← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction NAACL 2022
Lost in Distillation: A Case Study in Toxicity Modeling NAACL 2022
Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition AAAI 2022
Language Modelling via Learning to Rank AAAI 2022
ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation NIPS 2022
Federated Learning from Pre-Trained Models: A Contrastive Learning Approach NIPS 2022
Deep Model Reassembly NIPS 2022
On Efficient Online Imitation Learning via Classification NIPS 2022
Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models NIPS 2022
MobileFaceSwap: A Lightweight Framework for Video Face Swapping AAAI 2022
Single-Domain Generalized Object Detection in Urban Scene via Cyclic-Disentangled Self-Distillation CVPR 2022
Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation CVPR 2022
Norm-based Noisy Corpora Filtering and Refurbishing in Neural Machine Translation EMNLP 2022
Wider & Closer: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition EMNLP 2022
Evaluating Parameter Efficient Learning for Generation EMNLP 2022
Deeply Tensor Compressed Transformers for End-to-End Object Detection AAAI 2022
LGD: Label-Guided Self-Distillation for Object Detection AAAI 2022
Content-Variant Reference Image Quality Assessment via Knowledge Distillation AAAI 2022
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022
Sparse Teachers Can Be Dense with Knowledge EMNLP 2022
Not to Overfit or Underfit the Source Domains? An Empirical Study of Domain Generalization in Question Answering EMNLP 2022
Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model EMNLP 2022
Distill The Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation EMNLP 2022
Dreaming To Prune Image Deraining Networks CVPR 2022
Decoupled Multi-Task Learning With Cyclical Self-Regulation for Face Parsing CVPR 2022