← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

TAKE: Topic-shift Aware Knowledge sElection for Dialogue Generation COLING 2022
Attribute Injection for Pretrained Language Models: A New Benchmark and an Efficient Method COLING 2022
Combining Compressions for Multiplicative Size Scaling on Natural Language Tasks COLING 2022
Transferring Knowledge from Structure-aware Self-attention Language Model to Sequence-to-Sequence Semantic Parsing COLING 2022
Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation INTERSPEECH 2022
Multi-Scale Distillation from Multiple Graph Neural Networks AAAI 2022
Federated Class-Incremental Learning CVPR 2022
Diversifying Neural Dialogue Generation via Negative Distillation NAACL 2022
On the Use of External Data for Spoken Named Entity Recognition NAACL 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation NAACL 2022
Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation NAACL 2022
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation NAACL 2022
Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models NAACL 2022
Re2G: Retrieve, Rerank, Generate NAACL 2022
One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation NAACL 2022
Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics NAACL 2022
Knowledge Inheritance for Pre-trained Language Models NAACL 2022
Leaner and Faster: Two-Stage Model Compression for Lightweight Text-Image Retrieval NAACL 2022
Collective Relevance Labeling for Passage Retrieval NAACL 2022
Causal Distillation for Language Models NAACL 2022
Learning Cross-Lingual IR from an English Retriever NAACL 2022
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models NAACL 2022
Nearest Neighbor Knowledge Distillation for Neural Machine Translation NAACL 2022
FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction NAACL 2022
Multi-View Attention Transfer for Efficient Speech Enhancement INTERSPEECH 2022