← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Mixed Distillation Helps Smaller Language Models Reason Better EMNLP 2024
Divide-or-Conquer? Which Part Should You Distill Your LLM? EMNLP 2024
Privacy Evaluation Benchmarks for NLP Models EMNLP 2024
Evolutionary Contrastive Distillation for Language Model Alignment EMNLP 2024
Dual-teacher Knowledge Distillation for Low-frequency Word Translation EMNLP 2024
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning EMNLP 2024
AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models EMNLP 2024
PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning EMNLP 2024
DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs EMNLP 2024
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages EMNLP 2024
Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL EMNLP 2024
Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs EMNLP 2024
TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models EMNLP 2024
Fine-tuning Smaller Language Models for Question Answering over Financial Documents EMNLP 2024
LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement EMNLP 2024
Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model EMNLP 2024
“Seeing the Big through the Small”: Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations? EMNLP 2024
Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models with Personality-Indicative Data EMNLP 2024
Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning EMNLP 2024
MolTRES: Improving Chemical Language Representation Learning for Molecular Property Prediction EMNLP 2024
MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification EMNLP 2024
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation EMNLP 2024
Breaking ReLU Barrier: Generalized MoEfication for Dense Pretrained Models EMNLP 2024
KNN-Instruct: Automatic Instruction Construction with K Nearest Neighbor Deduction EMNLP 2024