Knowledge Distillation

HOICLIP: Efficient Knowledge Transfer for HOI Detection With Vision-Language Models CVPR 2023 How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives ACL 2023 IAG: Induction-Augmented Generation Framework for Answering Reasoning Questions EMNLP 2023 Improving Distantly Supervised Relation Extraction by Natural Language Inference AAAI 2023 Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcripts ACL 2023 Improving Knowledge Graph Completion with Generative Hard Negative Mining ACL 2023 Improving Simultaneous Machine Translation with Monolingual Data AAAI 2023 Incrementer: Transformer for Class-Incremental Semantic Segmentation With Knowledge Distillation Focusing on Old Class CVPR 2023 InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT EMNLP 2023 KD-DLGAN: Data Limited Image Generation via Knowledge Distillation CVPR 2023 Knowledge Amalgamation for Multi-Label Classification via Label Dependency Transfer AAAI 2023 Knowledge Distillation Approach for Efficient Internal Language Model Estimation Interspeech 2023 Knowledge Distillation based Degradation Estimation for Blind Super-Resolution ICLR 2023 Knowledge Distillation for 6D Pose Estimation by Aligning Distributions of Local Predictions CVPR 2023 Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data Interspeech 2023 Knowledge Distillation ≈ Label Smoothing: Fact or Fallacy? EMNLP 2023 Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation Interspeech 2023 L2T-DLN: Learning to Teach with Dynamic Loss Network NeurIPS 2023 Large Language Models Are Reasoning Teachers ACL 2023 LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillation EACL 2023 Learn From One Specialized Sub-Teacher: One-to-One Mapping for Feature-Based Knowledge Distillation EMNLP 2023 Learning to Imagine: Distillation-Based Interactive Context Exploitation for Dialogue State Tracking AAAI 2023 Learning to Learn from APIs: Black-Box Data-Free Meta-Learning ICML 2023 Learning To Retain While Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation CVPR 2023 Learn More for Food Recognition via Progressive Self-Distillation AAAI 2023