Knowledge Distillation

On the Impact of Knowledge Distillation for Model Interpretability ICML 2023 PartManip: Learning Cross-Category Generalizable Part Manipulation Policy From Point Cloud Observations CVPR 2023 Plug-and-Play Document Modules for Pre-trained Models ACL 2023 Plug-and-Play Knowledge Injection for Pre-trained Language Models ACL 2023 Progressive Ensemble Distillation: Building Ensembles for Efficient Inference NeurIPS 2023 ProKD: An Unsupervised Prototypical Knowledge Distillation Network for Zero-Resource Cross-Lingual Named Entity Recognition AAAI 2023 PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation EMNLP 2023 Propagating Knowledge Updates to LMs Through Distillation NeurIPS 2023 Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning ICML 2023 PuMer: Pruning and Merging Tokens for Efficient Vision Language Models ACL 2023 Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs ICML 2023 Quantized Feature Distillation for Network Quantization AAAI 2023 Random Teachers are Good Teachers ICML 2023 RD-Suite: A Benchmark for Ranking Distillation NeurIPS 2023 RefCLIP: A Universal Teacher for Weakly Supervised Referring Expression Comprehension CVPR 2023 Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning AAAI 2023 Rethinking Data-Free Quantization as a Zero-Sum Game AAAI 2023 Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression EMNLP 2023 Revisiting Temporal Modeling for CLIP-Based Image-to-Video Knowledge Transferring CVPR 2023 RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding ACL 2023 Scaffolding a Student to Instill Knowledge ICLR 2023 ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector CVPR 2023 Scaling Up Probabilistic Circuits by Latent Variable Distillation ICLR 2023 SCOTT: Self-Consistent Chain-of-Thought Distillation ACL 2023 Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translation AAAI 2023