Knowledge Distillation

PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection CVPR 2023 Preserving Linear Separability in Continual Learning by Backward Feature Projection CVPR 2023 Pretrained Bidirectional Distillation for Machine Translation ACL 2023 Privacy-Preserving Knowledge Transfer through Partial Parameter Sharing ACL 2023 Probabilistic Integration of Object Level Annotations in Chest X-Ray Classification WACV 2023 Probabilistic Knowledge Distillation of Face Ensembles CVPR 2023 Program Translation via Code Distillation EMNLP 2023 Progressive Ensemble Distillation: Building Ensembles for Efficient Inference NeurIPS 2023 ProKD: An Unsupervised Prototypical Knowledge Distillation Network for Zero-Resource Cross-Lingual Named Entity Recognition AAAI 2023 Prompting Neural Machine Translation with Translation Memories AAAI 2023 PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation EMNLP 2023 Protecting Language Generation Models via Invisible Watermarking ICML 2023 PROTEGE: Prompt-based Diverse Question Generation from Web Articles EMNLP 2023 QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D Object Detection ICCV 2023 Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs ICML 2023 Quantized Feature Distillation for Network Quantization AAAI 2023 Random Teachers are Good Teachers ICML 2023 RD-Suite: A Benchmark for Ranking Distillation NeurIPS 2023 ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models ACL 2023 RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation ACL 2023 Recyclable Tuning for Continual Pre-training ACL 2023 Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation Interspeech 2023 RefCLIP: A Universal Teacher for Weakly Supervised Referring Expression Comprehension CVPR 2023 Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement ICCV 2023 RelightableHands: Efficient Neural Relighting of Articulated Hand Models CVPR 2023