Knowledge Distillation

On Self-Distilling Graph Neural Network IJCAI 2021 On the distribution of penultimate activations of classification networks UAI 2021 OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World CVPR 2021 Papago’s Submission for the WMT21 Quality Estimation Shared Task EMNLP 2021 Parameter-Efficient Domain Knowledge Integration from Multiple Sources for Biomedical Pre-trained Language Models EMNLP 2021 Parameterized Knowledge Transfer for Personalized Federated Learning NeurIPS 2021 PDALN: Progressive Domain Adaptation over a Pre-trained Model for Low-Resource Cross-Domain Named Entity Recognition EMNLP 2021 Peer Collaborative Learning for Online Knowledge Distillation AAAI 2021 Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation ICCV 2021 Positive-Congruent Training: Towards Regression-Free Model Updates CVPR 2021 Positive-Unlabeled Data Purification in the Wild for Object Detection CVPR 2021 PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation Interspeech 2021 Practical One-Shot Federated Learning for Cross-Silo Setting IJCAI 2021 PRAL: A Tailored Pre-Training Model for Task-Oriented Dialog Generation ACL 2021 PREMERE: Meta-Reweighting via Self-Ensembling for Point-of-Interest Recommendation AAAI 2021 Probabilistic Model Distillation for Semantic Correspondence CVPR 2021 Progressive Network Grafting for Few-Shot Knowledge Distillation AAAI 2021 Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks AAAI 2021 PSSM-Distil: Protein Secondary Structure Prediction (PSSP) on Low-Quality PSSM by Knowledge Distillation with Contrastive Learning AAAI 2021 Question Generation for Adaptive Education ACL 2021 QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning NeurIPS 2021 RankDistil: Knowledge Distillation for Ranking AISTATS 2021 Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation WACV 2021 Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation CVPR 2021 Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates ACL 2021