← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

General Incremental Learning With Domain-Aware Categorical Representations CVPR 2022
Self-Distillation From the Last Mini-Batch for Consistency Regularization CVPR 2022
On Generalizing Beyond Domains in Cross-Domain Continual Learning CVPR 2022
Consistent Representation Learning for Continual Relation Extraction ACL 2022
Lifelong Person Re-identification by Pseudo Task Knowledge Preservation AAAI 2022
ADD: Frequency Attention and Multi-View Based Knowledge Distillation to Detect Low-Quality Compressed Deepfake Images AAAI 2022
Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective Distillation ACL 2022
Cross-Domain Adaptive Teacher for Object Detection CVPR 2022
Point-Level Region Contrast for Object Detection Pre-Training CVPR 2022
EyePAD++: A Distillation-Based Approach for Joint Eye Authentication and Presentation Attack Detection Using Periocular Images CVPR 2022
Asymmetric Temperature Scaling Makes Larger Networks Teach Well Again NIPS 2022
Kronecker Decomposition for GPT Compression ACL 2022
Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation ACL 2022
Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning CVPR 2022
SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training NIPS 2022
Dataset Distillation by Matching Training Trajectories CVPR 2022
Controllable Dynamic Multi-Task Architectures CVPR 2022
Attention Temperature Matters in Abstractive Summarization Distillation ACL 2022
Can Current Explainability Help Provide References in Clinical Notes to Support Humans Annotate Medical Codes? EMNLP 2022
Legal-Tech Open Diaries: Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models EMNLP 2022
X-Trans2Cap: Cross-Modal Knowledge Transfer Using Transformer for 3D Dense Captioning CVPR 2022
Fine-Tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning CVPR 2022
How Many Observations Are Enough? Knowledge Distillation for Trajectory Forecasting CVPR 2022
Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gaps AACL 2022
Modeling Document-level Temporal Structures for Building Temporal Dependency Graphs AACL 2022