knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Interweaving Memories of a Siamese Large Language Model AAAI 2025

Distilling Structured Rationale from Large Language Models to Small Language Models for Abstractive Summarization AAAI 2025

Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings EMNLP 2025

Training Deep Neural Networks with Virtual Smoothing Classes AAAI 2025

Multimodal Commonsense Knowledge Distillation for Visual Question Answering (Student Abstract) AAAI 2025

Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model AAAI 2025

Distillation versus Contrastive Learning: How to Train Your Rerankers IJCNLP 2025

BcQLM: Efficient Vision-Language Understanding with Distilled Q-Gated Cross-Modal Fusion EMNLP 2025

On the Analysis and Distillation of Emergent Outlier Properties in Pre-trained Language Models NAACL 2025

Fed-DFA: Federated Distillation for Heterogeneous Model Fusion Through the Adversarial Lens AAAI 2025

Is LLMs Hallucination Usable? LLM-based Negative Reasoning for Fake News Detection AAAI 2025

Turbo2K: Towards Ultra-Efficient and High-Quality 2K Video Synthesis ICCV 2025

Toward Efficient Data-Free Unlearning AAAI 2025

Can Students Beyond the Teacher? Distilling Knowledge from Teacher’s Bias AAAI 2025

SMDAF: A Scalable Sidewalk Material Data Acquisition Framework with Bidirectional Cross-Modal Knowledge Distillation WACV 2025

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts ACL 2025

Depth Any Event Stream: Enhancing Event-based Monocular Depth Estimation via Dense-to-Sparse Distillation ICCV 2025

Optimizing Quantized Diffusion Models via Distillation with Cross-Timestep Error Correction AAAI 2025

Best Practices for Distilling Large Language Models into BERT for Web Search Ranking COLING 2025

Howard University-AI4PC at SemEval-2025 Task 4: Unlearning Sensitive Content From Large Language Models Using Finetuning and Distillation for Selective Knowledge Removal SEMEVAL 2025

Text-Guided Fine-grained Counterfactual Inference for Short Video Fake News Detection AAAI 2025

EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation AAAI 2025

Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement AAAI 2025

3DM: Distill, Dynamic Drop, and Merge for Debiasing Multi-modal Large Language Models ACL 2025

Staged Knowledge Distillation Through Least-to-Most Prompting: Optimizing Teacher Guidance via Difficulty-Aware Training EMNLP 2025