Skip to content
conftrace_
2025 ICML ICML 2025

On Teacher Hacking in Language Model Distillation

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Daniil Tiapkin, Daniele Calandriello, Johan Ferret, Sarah Perrin, Nino Vieillard, Alexandre Rame, Mathieu Blondel

Topics

Machine Learning > Optimization & Theory > Learning Theory Deep Learning > Architectures > Transformers Artificial Intelligence > Core AI > Large Language Models

Keywords

data diversity ground-truth distribution distillation online data generation oracle lm teacher hacking
Download PDF