← Application Areas
Machine Learning › Application Areas ›

Knowledge Distillation

2907 directly classified papers

Papers per year

Papers

C2KD: Bridging the Modality Gap for Cross-Modal Knowledge Distillation CVPR 2024
UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory CVPR 2024
Querying as Prompt: Parameter-Efficient Learning for Multimodal Language Model CVPR 2024
Mentor-KD: Making Small Language Models Better Multi-step Reasoners EMNLP 2024
D3ETR: Decoder Distillation for Detection Transformer IJCAI 2024
GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient Cloud-edge Collaboration LLM Deployment ACL 2024
LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement EMNLP 2024
FreeKD: Knowledge Distillation via Semantic Frequency Prompt CVPR 2024
STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models ACL 2024
PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor CVPR 2024
Using Early Readouts To Mediate Featural Bias in Distillation WACV 2024
Exploiting CLIP for Zero-Shot HOI Detection Requires Knowledge Distillation at Multiple Levels WACV 2024
ProS: Facial Omni-Representation Learning via Prototype-Based Self-Distillation WACV 2024
On the Surprising Effectiveness of Attention Transfer for Vision Transformers NIPS 2024
KALM: Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts NIPS 2024
Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation RSS 2024
Aligning Logits Generatively for Principled Black-Box Knowledge Distillation CVPR 2024
Divide-or-Conquer? Which Part Should You Distill Your LLM? EMNLP 2024
Mixed Distillation Helps Smaller Language Models Reason Better EMNLP 2024
RaFe: Ranking Feedback Improves Query Rewriting for RAG EMNLP 2024
Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024
Better Alignment with Instruction Back-and-Forth Translation EMNLP 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model EMNLP 2024
“Seeing the Big through the Small”: Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations? EMNLP 2024
Aligning Large Language Models by On-Policy Self-Judgment ACL 2024