← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation ACL 2022
Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation NIPS 2022
Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively NIPS 2022
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers NIPS 2022
Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction ACL 2022
Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts NIPS 2022
A Model-agnostic Data Manipulation Method for Persona-based Dialogue Generation ACL 2022
Target-Relevant Knowledge Preservation for Multi-Source Domain Adaptive Object Detection CVPR 2022
Aligned Weight Regularizers for Pruning Pretrained Neural Networks ACL 2022
When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation ACL 2022
Training Spiking Neural Networks with Local Tandem Learning NIPS 2022
Deep Compression of Pre-trained Transformer Models NIPS 2022
Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation ACL 2022
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm ACL 2022
E-LANG: Energy-Based Joint Inferencing of Super and Swift Language Models ACL 2022
Knowledge-Augmented Methods for Natural Language Processing ACL 2022
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation ACL 2022
GenCompareSum: a hybrid unsupervised summarization method using salience ACL 2022
Compression of Generative Pre-trained Language Models via Quantization ACL 2022
Structured Pruning Learns Compact and Accurate Models ACL 2022
Multi-Granularity Structural Knowledge Distillation for Language Model Compression ACL 2022
Decomposing NeRF for Editing via Feature Field Distillation NIPS 2022
FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models NAACL 2022
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework NAACL 2022
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation NAACL 2022