Knowledge Distillation

AdapterFusion: Non-Destructive Task Composition for Transfer Learning EACL 2021 Adjunct-Emeritus Distillation for Semi-Supervised Language Model Adaptation Interspeech 2021 Adversarial Teacher-Student Representation Learning for Domain Generalization NeurIPS 2021 AGKD-BML: Defense Against Adversarial Attack by Attention Guided Knowledge Distillation and Bi-Directional Metric Learning ICCV 2021 A Hands-On Comparison of DNNs for Dialog Separation Using Transfer Learning from Music Source Separation Interspeech 2021 A Lightweight Framework for Online Voice Activity Detection in the Wild Interspeech 2021 AlphaNet: Improved Training of Supernets with Alpha-Divergence ICML 2021 ALP-KD: Attention-Based Layer Projection for Knowledge Distillation AAAI 2021 Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning ICCV 2021 Amalgamating Knowledge From Heterogeneous Graph Neural Networks CVPR 2021 Analyzing the Confidentiality of Undistillable Teachers in Knowledge Distillation NeurIPS 2021 An Iterative Multi-Knowledge Transfer Network for Aspect-Based Sentiment Analysis EMNLP 2021 Annealing Knowledge Distillation EACL 2021 An Overview of Uncertainty Calibration for Text Classification and the Role of Distillation ACL 2021 ANVITA Machine Translation System for WAT 2021 MultiIndicMT Shared Task ACL 2021 A Peek Into the Reasoning of Neural Networks: Interpreting With Structural Visual Concepts CVPR 2021 A statistical perspective on distillation ICML 2021 Asynchronous Teacher Guided Bit-wise Hard Mining for Online Hashing AAAI 2021 Audio-Oriented Multimodal Machine Comprehension via Dynamic Inter- and Intra-modality Attention AAAI 2021 Automatic Construction of Enterprise Knowledge Base EMNLP 2021 Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation Mechanism ACL 2021 Automatic Mixed-Precision Quantization Search of BERT IJCAI 2021 AUTOSUMM: Automatic Model Creation for Text Summarization EMNLP 2021 Better Neural Machine Translation by Extracting Linguistic Information from BERT EACL 2021 Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference EMNLP 2021