conftrace_
← Keywords

knowledge distillation

3725 papers

Explore in graph

Also known as

KD

Co-occurring keywords

model compression (3302) large language model (13587) transfer learning (5449) domain adaptation (4595) representation learning (6206) neural network (6616) language model (4599) catastrophic forgetting (958) continual learning (1181) contrastive learning (4032)

Papers

Distilling Object Detectors with Feature Richness NIPS 2021
MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps NIPS 2021
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning NIPS 2021
Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme NIPS 2021
Analyzing the Confidentiality of Undistillable Teachers in Knowledge Distillation NIPS 2021
Aligned Structured Sparsity Learning for Efficient Image Super-Resolution NIPS 2021
Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translation NAACL 2021
Self-training Improves Pre-training for Natural Language Understanding NAACL 2021
AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling CVPR 2021
There Is More Than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking With Sound by Distilling Multimodal Knowledge CVPR 2021
Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation CVPR 2021
ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques AAAI 2021
RankDistil: Knowledge Distillation for Ranking AISTATS 2021
Isotonic Data Augmentation for Knowledge Distillation IJCAI 2021
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation IJCAI 2021
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor ACL 2021
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation ACL 2021
Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains ACL 2021
Improving Streaming Transformer Based ASR Under a Framework of Self-Supervised Learning INTERSPEECH 2021
Self-Adaptive Distillation for Multilingual Speech Recognition: Leveraging Student Independence INTERSPEECH 2021
A Lightweight Framework for Online Voice Activity Detection in the Wild INTERSPEECH 2021
Adjunct-Emeritus Distillation for Semi-Supervised Language Model Adaptation INTERSPEECH 2021
CoDERT: Distilling Encoder Representations with Co-Learning for Transducer-Based Speech Recognition INTERSPEECH 2021
Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification INTERSPEECH 2021
Multi-Level Transfer Learning from Near-Field to Far-Field Speaker Verification INTERSPEECH 2021