Knowledge Distillation

Multimodal Machine Translation for Low-Resource Indic Languages: A Chain-of-Thought Approach Using Large Language Models EMNLP 2024 Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation EMNLP 2024 Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation ACL 2024 Multistep Distillation of Diffusion Models via Moment Matching NeurIPS 2024 MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval CVPR 2024 Navigating Continual Test-time Adaptation with Symbiosis Knowledge IJCAI 2024 NAYER: Noisy Layer Data Generation for Efficient and Effective Data-free Knowledge Distillation CVPR 2024 Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent EMNLP 2024 NetInfoF Framework: Measuring and Exploiting Network Usable Information ICLR 2024 NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge NAACL 2024 No Head Left Behind – Multi-Head Alignment Distillation for Transformers AAAI 2024 NOLA: Compressing LoRA using Linear Combination of Random Basis ICLR 2024 Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models ACL 2024 OD-DETR: Online Distillation for Stabilizing Training of Detection Transformer IJCAI 2024 Offline Behavior Distillation NeurIPS 2024 Offline-to-Online Knowledge Distillation for Video Instance Segmentation WACV 2024 Okay, Let’s Do This! Modeling Event Coreference with Generated Rationales and Knowledge Distillation NAACL 2024 OMG-QA: Building Open-Domain Multi-Modal Generative Question Answering Systems EMNLP 2024 Omnipotent Distillation with LLMs for Weakly-Supervised Natural Language Video Localization: When Divergence Meets Consistency AAAI 2024 One-shot Federated Learning via Synthetic Distiller-Distillate Communication NeurIPS 2024 One-step Diffusion with Distribution Matching Distillation CVPR 2024 On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion NeurIPS 2024 Online Knowledge Distillation of Decoder-Only Large Language Models for Efficient Speech Recognition Interspeech 2024 On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes ICLR 2024 On the Ability of Developers' Training Data Preservation of Learnware NeurIPS 2024