Skip to content
conftrace_

capacity gap

8 papers

Explore in graph

Co-occurring keywords · papers mentioning both

knowledge distillation (7) model compression (7) teacher-student learning (2) student-teacher learning (2) language model (2) transfer learning (2) end-to-end speech translation (1) checkpoint selection (1) capacity-gap problem (1) teacher network (1)

Papers

Beyond One-Step Distillation: Bridging the Capacity Gap in Small Language Models via Multi-Step Knowledge Transfer EACL 2026 Adaptive Dual Guidance Knowledge Distillation AAAI 2025 A Good Teacher Adapts Their Knowledge for Distillation ICCV 2025 Towards the Law of Capacity Gap in Distilling Language Models ACL 2025 Lifting the Curse of Capacity Gap in Distilling Language Models ACL 2023 Modality Adaption or Regularization? A Case Study on End-to-End Speech Translation ACL 2023 Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization EMNLP 2022 Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher COLING 2022