knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

A Lightweight Mixture-of-Experts Neural Machine Translation Model with Stage-wise Training Strategy NAACL 2024

Towards an On-device Agent for Text Rewriting NAACL 2024

UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle NAACL 2024

DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation NAACL 2024

A Novel Two-step Fine-tuning Framework for Transfer Learning in Low-Resource Neural Machine Translation NAACL 2024

Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer NAACL 2024

Normalizing without Modernizing: Keeping Historical Wordforms of Middle French while Reducing Spelling Variants NAACL 2024

Efficient Citer: Tuning Large Language Models for Enhanced Answer Quality and Verification NAACL 2024

NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge NAACL 2024

Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget NAACL 2024

Leros: Learning Explicit Reasoning on Synthesized Data for Commonsense Question Answering COLING 2024

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward COLING 2024

MinT: Boosting Generalization in Mathematical Reasoning via Multi-view Fine-tuning COLING 2024

Multilingual Brain Surgeon: Large Language Models Can Be Compressed Leaving No Language behind COLING 2024

Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion AAAI 2024

MIND: Multi-Task Incremental Network Distillation AAAI 2024

To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation ACL 2024

Label Attentive Distillation for GNN-Based Graph Classification AAAI 2024

Tail-STEAK: Improve Friend Recommendation for Tail Users via Self-Training Enhanced Knowledge Distillation AAAI 2024

Chinese Spoken Named Entity Recognition in Real-world Scenarios: Dataset and Approaches ACL 2024

NICT’s Cascaded and End-To-End Speech Translation Systems using Whisper and IndicTrans2 for the Indic Task ACL 2024

For the Misgendered Chinese in Gender Bias Research: Multi-Task Learning with Knowledge Distillation for Pinyin Name Gender Prediction IJCAI 2024

Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation IJCAI 2024

Guiding Clinical Reasoning with Large Language Models via Knowledge Seeds IJCAI 2024

Ungeneralizable Examples CVPR 2024