Knowledge Distillation

Masked Distillation with Receptive Tokens ICLR 2023 Masked Images Are Counterfactual Samples for Robust Fine-Tuning CVPR 2023 Masked Modeling Duo for Speech: Specializing General-Purpose Audio Representation to Speech using Denoising Distillation Interspeech 2023 Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Models ACL 2023 MCC-KD: Multi-CoT Consistent Knowledge Distillation EMNLP 2023 mCLIP: Multilingual CLIP via Cross-lingual Transfer ACL 2023 MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition ICCV 2023 MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training for X-ray Diagnosis ICCV 2023 MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentation ICCV 2023 MEID: Mixture-of-Experts with Internal Distillation for Long-Tailed Video Recognition AAAI 2023 Merging Generated and Retrieved Knowledge for Open-Domain QA EMNLP 2023 Meta Knowledge Condensation for Federated Learning ICLR 2023 Meta-prediction Model for Distillation-Aware NAS on Unseen Datasets ICLR 2023 Metis: Understanding and Enhancing In-Network Regular Expressions NeurIPS 2023 MetricGAN-OKD: Multi-Metric Optimization of MetricGAN via Online Knowledge Distillation for Speech Enhancement ICML 2023 MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers EACL 2023 MixFormerV2: Efficient Fully Transformer Tracking NeurIPS 2023 Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models EMNLP 2023 Mixture Uniform Distribution Modeling and Asymmetric Mix Distillation for Class Incremental Learning AAAI 2023 MLFMF: Data Sets for Machine Learning for Mathematical Formalization NeurIPS 2023 MMANet: Margin-Aware Distillation and Modality-Aware Regularization for Incomplete Multimodal Learning CVPR 2023 MobileVOS: Real-Time Video Object Segmentation Contrastive Learning Meets Knowledge Distillation CVPR 2023 Model Conversion via Differentially Private Data-Free Distillation IJCAI 2023 Models Teaching Models: Improving Model Accuracy with Slingshot Learning EACL 2023 Mod-Squad: Designing Mixtures of Experts As Modular Multi-Task Learners CVPR 2023