Knowledge Distillation

Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Models CVPR 2024 Accelerating Dataset Distillation via Model Augmentation CVPR 2023 Accelerating Multiple Intent Detection and Slot Filling via Targeted Knowledge Distillation EMNLP 2023 AdapterDistillation: Non-Destructive Task Composition with Knowledge Distillation EMNLP 2023 Adaptive Contrastive Knowledge Distillation for BERT Compression ACL 2023 Adaptive Mixing of Auxiliary Losses in Supervised Learning AAAI 2023 AD-KD: Attribution-Driven Knowledge Distillation for Language Model Compression ACL 2023 A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation ICCV 2023 A Multiple-Teacher Pruning Based Self-Distillation (MT-PSD) Approach to Model Compression for Audio-Visual Wake Word Spotting Interspeech 2023 An Efficient and Noise-Robust Audiovisual Encoder for Audiovisual Speech Recognition Interspeech 2023 An Ensemble Distillation Framework for Sentence Embeddings with Multilingual Round-Trip Translation AAAI 2023 Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method ACL 2023 Are Large Kernels Better Teachers than Transformers for ConvNets? ICML 2023 A Simple Concatenation can Effectively Improve Speech Translation ACL 2023 A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models ACL 2023 A Study on the Efficiency and Generalization of Light Hybrid Retrievers ACL 2023 Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection AAAI 2023 Augmentation-Free Dense Contrastive Knowledge Distillation for Efficient Semantic Segmentation NeurIPS 2023 Auxiliary Modality Learning with Generalized Curriculum Distillation ICML 2023 Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models EMNLP 2023 Better Language Models of Code through Self-Improvement ACL 2023 Better Simultaneous Translation with Monotonic Knowledge Distillation ACL 2023 Bit-Shrinking: Limiting Instantaneous Sharpness for Improving Post-Training Quantization CVPR 2023 Boosting Accuracy and Robustness of Student Models via Adaptive Adversarial Distillation CVPR 2023 Boosting Graph Neural Networks via Adaptive Knowledge Distillation AAAI 2023