Why distillation helps: a statistical perspective
Knowledge distillation is a technique for improving the performance of a simple "student" model by replacing its one-hot training labels with a distribution over labels obtained from a complex "teacher" model. While this simple approach has proven widely effective, a basic question remains unresolve...
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Artículo |
| Publicado: |
2020
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2005.10419 http://arxiv.org/abs/2005.10419 |
| Etiquetas: |
Sin Etiquetas, Sea el primero en etiquetar este registro!
|