conftrace_
← Keywords

knowledge distillation

3725 papers

Explore in graph

Also known as

KD

Co-occurring keywords

model compression (3302) large language model (13587) transfer learning (5449) domain adaptation (4595) representation learning (6206) neural network (6616) language model (4599) catastrophic forgetting (958) continual learning (1181) contrastive learning (4032)

Papers

Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval EMNLP 2022
Weak-shot Semantic Segmentation via Dual Similarity Transfer NIPS 2022
Contact-Distil: Boosting Low Homologous Protein Contact Map Prediction by Self-Supervised Distillation AAAI 2022
Conformer with dual-mode chunked attention for joint online and offline ASR INTERSPEECH 2022
Knowledge distillation for In-memory keyword spotting model INTERSPEECH 2022
Data-Free Adversarial Knowledge Distillation for Graph Neural Networks IJCAI 2022
Can Functional Transfer Methods Capture Simple Inductive Biases? AISTATS 2022
TRILLsson: Distilled Universal Paralinguistic Speech Representations INTERSPEECH 2022
Device-Directed Speech Detection: Regularization via Distillation for Weakly-Supervised Models INTERSPEECH 2022
Adversarial Knowledge Distillation For Robust Spoken Language Understanding INTERSPEECH 2022
Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search CVPR 2022
Multi-stage Progressive Compression of Conformer Transducer for On-device Speech Recognition INTERSPEECH 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT INTERSPEECH 2022
Multi-Level Branched Regularization for Federated Learning ICML 2022
Rich Feature Construction for the Optimization-Generalization Dilemma ICML 2022
XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding ACL 2022
The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes NIPS 2022
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection EMNLP 2022
Distillation-Resistant Watermarking for Model Protection in NLP EMNLP 2022
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging EMNLP 2022
What Do Compressed Multilingual Machine Translation Models Forget? EMNLP 2022
Partially-Random Initialization: A Smoking Gun for Binarization Hypothesis of BERT EMNLP 2022
Quadapter: Adapter for GPT-2 Quantization EMNLP 2022
ExpertPLM: Pre-training Expert Representation for Expert Finding EMNLP 2022
Plug and Play Knowledge Distillation for kNN-LM with External Logits AACL 2022