Knowledge Distillation

Mentor-KD: Making Small Language Models Better Multi-step Reasoners EMNLP 2024 MGCL: Multi-Granularity Clue Learning for Emotion-Cause Pair Extraction via Cross-Grained Knowledge Distillation EMNLP 2024 MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding EMNLP 2024 MIND: Multi-Task Incremental Network Distillation AAAI 2024 Mind’s Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models NAACL 2024 MiniLLM: Knowledge Distillation of Large Language Models ICLR 2024 Minimal Distillation Schedule for Extreme Language Model Compression EACL 2024 Mirage: Model-agnostic Graph Distillation for Graph Classification ICLR 2024 MISA: MIning Saliency-Aware Semantic Prior for Box Supervised Instance Segmentation IJCAI 2024 Mitigating Language-Level Performance Disparity in mPLMs via Teacher Language Selection and Cross-lingual Self-Distillation NAACL 2024 Mixed Distillation Helps Smaller Language Models Reason Better EMNLP 2024 Mixture of LoRA Experts ICLR 2024 Mixture-of-LoRAs: An Efficient Multitask Tuning Method for Large Language Models COLING 2024 MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training CVPR 2024 MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts AAAI 2024 MoDE-CoTD: Chain-of-Thought Distillation for Complex Reasoning Tasks with Mixture of Decoupled LoRA-Experts COLING 2024 Model Editing by Standard Fine-Tuning ACL 2024 Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks NeurIPS 2024 MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning EMNLP 2024 MOFI: Learning Image Representations from Noisy Entity Annotated Images ICLR 2024 MTA4DPR: Multi-Teaching-Assistants Based Iterative Knowledge Distillation for Dense Passage Retrieval EMNLP 2024 MTLoRA: Low-Rank Adaptation Approach for Efficient Multi-Task Learning CVPR 2024 MT-PATCHER: Selective and Extendable Knowledge Distillation from Large Language Models for Machine Translation NAACL 2024 Multi-Expert Distillation for Few-Shot Coordination (Student Abstract) AAAI 2024 Multimodal Large Language Models with Fusion Low Rank Adaptation for Device Directed Speech Detection Interspeech 2024