← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Learning Answer Generation using Supervision from Automatic Question Answering Evaluators ACL 2023
Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models ACL 2023
Revisiting Commonsense Reasoning in Machine Translation: Training, Evaluation and Challenge ACL 2023
A Simple Concatenation can Effectively Improve Speech Translation ACL 2023
How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives ACL 2023
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method ACL 2023
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference ACL 2023
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models ACL 2023
I2R’s End-to-End Speech Translation System for IWSLT 2023 Offline Shared Task ACL 2023
Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition EMNLP 2023
Length-Adaptive Distillation: Customizing Small Language Model for Dynamic Token Pruning EMNLP 2023
Application of Knowledge Distillation to Multi-Task Speech Representation Learning INTERSPEECH 2023
Knowledge Distillation on Joint Task End-to-End Speech Translation INTERSPEECH 2023
FerKD: Surgical Label Adaptation for Efficient Distillation ICCV 2023
Connective Prediction for Implicit Discourse Relation Recognition via Knowledge Distillation ACL 2023
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue ACL 2023
Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels ICCV 2023
Bayesian Optimization Meets Self-Distillation ICCV 2023
BiViT: Extremely Compressed Binary Vision Transformers ICCV 2023
Learning to Distill Global Representation for Sparse-View CT ICCV 2023
Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors ICCV 2023
SEFD: Learning to Distill Complex Pose and Occlusion ICCV 2023
Distill n’ Explain: explaining graph neural networks using simple surrogates AISTATS 2023
A Teacher-Student Approach for Extracting Informative Speaker Embeddings From Speech Mixtures INTERSPEECH 2023
Soft Target-Enhanced Matching Framework for Deep Entity Matching AAAI 2023