knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation ACL 2025

Imputation-free Incomplete Multi-view Clustering via Knowledge Distillation IJCAI 2025

DCA: Dividing and Conquering Amnesia in Incremental Object Detection AAAI 2025

Synthetic Data is an Elegant GIFT for Continual Vision-Language Models CVPR 2025

3DM: Distill, Dynamic Drop, and Merge for Debiasing Multi-modal Large Language Models ACL 2025

Pruning General Large Language Models into Customized Expert Models ACL 2025

Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation AAAI 2025

OGP-Net: Optical Guidance Meets Pixel-Level Contrastive Distillation for Robust Multi-Modal and Missing Modality Segmentation AAAI 2025

Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models EMNLP 2025

Advancing Multiple Instance Learning with Continual Learning for Whole Slide Imaging CVPR 2025

DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models ACL 2025

Shy-hunyuan-MT at WMT25 General Machine Translation Shared Task EMNLP 2025

Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction ACL 2025

LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs ACL 2025

Large Language Model Meets Graph Neural Network in Knowledge Distillation AAAI 2025

Memorization Inheritance in Sequence-Level Knowledge Distillation for Neural Machine Translation ACL 2025

AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning ACL 2025

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation ACL 2025

ATLANTIS: Weak-to-Strong Learning via Importance Sampling ACL 2025

Recall with Reasoning: Chain-of-Thought Distillation for Mamba’s Long-Context Memory and Extrapolation EMNLP 2025

Distilling an End-to-End Voice Assistant Without Instruction Training Data ACL 2025

BLADE: Enhancing Black-Box Large Language Models with Small Domain-Specific Models AAAI 2025

Building a Family of Data Augmentation Models for Low-cost LLM Fine-tuning on the Cloud COLING 2025

Capture the Key in Reasoning to Enhance CoT Distillation Generalization ACL 2025

Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning ACL 2025