Knowledge Distillation

Learning Distilled Collaboration Graph for Multi-Agent Perception NeurIPS 2021 Learning From the Master: Distilling Cross-Modal Advanced Knowledge for Lip Reading CVPR 2021 Learning Light-Weight Translation Models from Deep Transformer AAAI 2021 Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution CVPR 2021 Learning Student-Friendly Teacher Networks for Knowledge Distillation NeurIPS 2021 Learning Student Networks in the Wild CVPR 2021 Learning to Augment for Data-scarce Domain BERT Knowledge Distillation AAAI 2021 Learning to Distill Convolutional Features Into Compact Local Descriptors WACV 2021 Learning to Reweight with Deep Interactions AAAI 2021 Learn-to-Share: A Hardware-friendly Transfer Learning Framework Exploiting Computation and Parameter Sharing ICML 2021 Limitations of Knowledge Distillation for Zero-shot Transfer Learning EMNLP 2021 Lipschitz Continuity Guided Knowledge Distillation ICCV 2021 LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding AAAI 2021 Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation ACL 2021 Matching Distributions between Model and Data: Cross-domain Knowledge Distillation for Unsupervised Domain Adaptation ACL 2021 MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation ACL 2021 Membership Privacy for Machine Learning Models Through Knowledge Transfer AAAI 2021 Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains ACL 2021 MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps NeurIPS 2021 MixKD: Towards Efficient Distillation of Large-scale Language Models ICLR 2021 MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing ICCV 2021 Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT EMNLP 2021 Modality-specific Distillation NAACL 2021 Model Distillation for Revenue Optimization: Interpretable Personalized Pricing ICML 2021 Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data NeurIPS 2021