← Application Areas
Machine Learning › Application Areas ›

Knowledge Distillation

2907 directly classified papers

Papers per year

Papers

Bayesian Optimization Meets Self-Distillation ICCV 2023
MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition ICCV 2023
Semi-Supervised Learning via Weight-Aware Distillation under Class Distribution Mismatch ICCV 2023
CLRL-Tuning: A Novel Continual Learning Approach for Automatic Speech Recognition INTERSPEECH 2023
MEID: Mixture-of-Experts with Internal Distillation for Long-Tailed Video Recognition AAAI 2023
Boosting Graph Neural Networks via Adaptive Knowledge Distillation AAAI 2023
Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting It into MLPs: An Effective GNN-to-MLP Distillation Framework AAAI 2023
Lifelong Variational Autoencoder via Online Adversarial Expansion Strategy AAAI 2023
Continual Variational Autoencoder via Continual Generative Knowledge Distillation AAAI 2023
Cross-Modal Distillation for Speaker Recognition AAAI 2023
DisGUIDE: Disagreement-Guided Data-Free Model Extraction AAAI 2023
Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher AAAI 2023
ProKD: An Unsupervised Prototypical Knowledge Distillation Network for Zero-Resource Cross-Lingual Named Entity Recognition AAAI 2023
An Ensemble Distillation Framework for Sentence Embeddings with Multilingual Round-Trip Translation AAAI 2023
HAPI Explorer: Comprehension, Discovery, and Explanation on History of ML APIs AAAI 2023
The economic trade-offs of large language models: A case study ACL 2023
Class Lifelong Learning for Intent Detection via Structure Consolidation Networks ACL 2023
TADA : Task Agnostic Dialect Adapters for English ACL 2023
TempLM: Distilling Language Models into Template-Based Generators ACL 2023
garNER at SemEval-2023: Simplified Knowledge Augmentation for Multilingual Complex Named Entity Recognition ACL 2023
Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models ACL 2023
How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives ACL 2023
Residual Prompt Tuning: improving prompt tuning with residual reparameterization ACL 2023
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method ACL 2023
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference ACL 2023