DA-KD: Difficulty-Aware Knowledge Distillation for Efficient Large Language Models

Changyi He; Yifu Ding; Jinyang Guo; Ruihao Gong; Haotong Qin; Xianglong Liu

2025 ICML ICML 2025

DA-KD: Difficulty-Aware Knowledge Distillation for Efficient Large Language Models

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Changyi He, Yifu Ding, Jinyang Guo, Ruihao Gong, Haotong Qin, Xianglong Liu

Topics

Machine Learning > Optimization & Theory > Optimization Machine Learning > Application Areas > Efficient Computing Artificial Intelligence > Core AI > Large Language Models

Keywords

knowledge distillation large language model training cost compression bidirectional discrepancy loss difficulty-aware

Download PDF