Knowledge Distillation

Layer Attack Unlearning: Fast and Accurate Machine Unlearning via Layer Level Attack and Knowledge Distillation AAAI 2024 Layer Compression of Deep Networks with Straight Flows AAAI 2024 LC4EE: LLMs as Good Corrector for Event Extraction ACL 2024 Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL EMNLP 2024 Let All Be Whitened: Multi-Teacher Distillation for Efficient Visual Retrieval AAAI 2024 Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification CVPR 2024 Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection EMNLP 2024 LLM-QAT: Data-Free Quantization Aware Training for Large Language Models ACL 2024 LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement EMNLP 2024 LLM See, LLM Do: Leveraging Active Inheritance to Target Non-Differentiable Objectives EMNLP 2024 Local-Global Multi-Modal Distillation for Weakly-Supervised Temporal Video Grounding AAAI 2024 LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks ACL 2024 LoRA-GA: Low-Rank Adaptation with Gradient Approximation NeurIPS 2024 LoRA Meets Dropout under a Unified Framework ACL 2024 LSTKC: Long Short-Term Knowledge Consolidation for Lifelong Person Re-identification AAAI 2024 M2SD:Multiple Mixing Self-Distillation for Few-Shot Class-Incremental Learning AAAI 2024 M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation ACL 2024 MACE: Mass Concept Erasure in Diffusion Models CVPR 2024 Memory-Augmenting Decoder-Only Language Models through Encoders (Student Abstract) AAAI 2024 Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging EMNLP 2024 MGCL: Multi-Granularity Clue Learning for Emotion-Cause Pair Extraction via Cross-Grained Knowledge Distillation EMNLP 2024 MIND: Multi-Task Incremental Network Distillation AAAI 2024 MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts AAAI 2024 MoDE-CoTD: Chain-of-Thought Distillation for Complex Reasoning Tasks with Mixture of Decoupled LoRA-Experts COLING 2024 Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue EMNLP 2024