knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets NIPS 2023

Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation ICCV 2023

Exploring Open-Vocabulary Semantic Segmentation from CLIP Vision Encoder Distillation Only ICCV 2023

Alleviating Catastrophic Forgetting of Incremental Object Detection via Within-Class and Between-Class Knowledge Distillation ICCV 2023

Self-Improvement of Non-autoregressive Model via Sequence-Level Distillation EMNLP 2023

torchdistill Meets Hugging Face Libraries for Reproducible, Coding-Free Deep Learning Studies: A Case Study on NLP EMNLP 2023

Periodically Exchange Teacher-Student for Source-Free Object Detection ICCV 2023

Compressed MoE ASR Model Based on Knowledge Distillation and Quantization INTERSPEECH 2023

Knowledge Distillation Performs Partial Variance Reduction NIPS 2023

The student becomes the master: Outperforming GPT3 on Scientific Factual Error Correction EMNLP 2023

Learning Easily Updated General Purpose Text Representations with Adaptable Task-Specific Prefix EMNLP 2023

PivotFEC: Enhancing Few-shot Factual Error Correction with a Pivot Task Approach using Large Language Models EMNLP 2023

Don’t Add, don’t Miss: Effective Content Preserving Generation from Pre-Selected Text Spans EMNLP 2023

Tuna: Instruction Tuning using Feedback from Large Language Models EMNLP 2023

Learning Semantic Role Labeling from Compatible Label Sequences EMNLP 2023

Robust distillation for worst-class performance: on the interplay between teacher and student objectives UAI 2023

Students Parrot Their Teachers: Membership Inference on Model Distillation NIPS 2023

Distillation Strategies for Discriminative Speech Recognition Rescoring INTERSPEECH 2023

Collaborative Learning via Prediction Consensus NIPS 2023

Localized Symbolic Knowledge Distillation for Visual Commonsense Models NIPS 2023

Logit-based ensemble distribution distillation for robust autoregressive sequence uncertainties UAI 2023

Learn and Sample Together: Collaborative Generation for Graphic Design Layout IJCAI 2023

Are Large Kernels Better Teachers than Transformers for ConvNets? ICML 2023

Random Teachers are Good Teachers ICML 2023

NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning ICML 2023