← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

What Do Compressed Multilingual Machine Translation Models Forget? EMNLP 2022
Partially-Random Initialization: A Smoking Gun for Binarization Hypothesis of BERT EMNLP 2022
Quadapter: Adapter for GPT-2 Quantization EMNLP 2022
ExpertPLM: Pre-training Expert Representation for Expert Finding EMNLP 2022
Plug and Play Knowledge Distillation for kNN-LM with External Logits AACL 2022
Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks AACL 2022
Post-Training with Interrogative Sentences for Enhancing BART-based Korean Question Generator AACL 2022
MemSR: Training Memory-efficient Lightweight Model for Image Super-Resolution ICML 2022
Resilient and Communication Efficient Learning for Heterogeneous Federated Systems ICML 2022
PILE: Pairwise Iterative Logits Ensemble for Multi-Teacher Labeled Distillation EMNLP 2022
QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation EMNLP 2022
Distilling Multilingual Transformers into CNNs for Scalable Intent Classification EMNLP 2022
An Effective Post-training Embedding Binarization Approach for Fast Online Top-K Passage Matching AACL 2022
Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks EMNLP 2022
Production federated keyword spotting via distillation, filtering, and joint federated-centralized training INTERSPEECH 2022
Meta-Learning Adaptive Knowledge Distillation for Efficient Biomedical Natural Language Processing AACL 2022
UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation IJCAI 2022
VIRT: Improving Representation-based Text Matching via Virtual Interaction EMNLP 2022
Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation ACL 2022
DiPCAN: Distilling Privileged Information for Crowd-Aware Navigation RSS 2022
Regularizing Brain Age Prediction via Gated Knowledge Distillation MIDL 2022
Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks IJCNLP 2022
Modeling Document-level Temporal Structures for Building Temporal Dependency Graphs IJCNLP 2022
Plug and Play Knowledge Distillation for kNN-LM with External Logits IJCNLP 2022
Keyword Spotting with Synthetic Data using Heterogeneous Knowledge Distillation INTERSPEECH 2022