knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Distilling Efficient Language-Specific Models for Cross-Lingual Transfer ACL 2023

MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition ICCV 2023

Semi-Supervised Learning via Weight-Aware Distillation under Class Distribution Mismatch ICCV 2023

Knowledge-Aware Federated Active Learning with Non-IID Data ICCV 2023

The MineTrans Systems for IWSLT 2023 Offline Speech Translation and Speech-to-Speech Translation Tasks ACL 2023

Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement ICCV 2023

Knowledge-Spreader: Learning Semi-Supervised Facial Action Dynamics by Consistifying Knowledge Granularity ICCV 2023

ORC: Network Group-based Knowledge Distillation using Online Role Change ICCV 2023

Fine-tuning Audio Spectrogram Transformer with Task-aware Adapters for Sound Event Detection INTERSPEECH 2023

Distilling knowledge from Gaussian process teacher to neural network student INTERSPEECH 2023

A Multiple-Teacher Pruning Based Self-Distillation (MT-PSD) Approach to Model Compression for Audio-Visual Wake Word Spotting INTERSPEECH 2023

Sequence-Level Knowledge Distillation for Class-Incremental End-to-End Spoken Language Understanding INTERSPEECH 2023

Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction ACL 2023

Better Language Models of Code through Self-Improvement ACL 2023

EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning ACL 2023

CLRL-Tuning: A Novel Continual Learning Approach for Automatic Speech Recognition INTERSPEECH 2023

DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation ICCV 2023

A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance ICCV 2023

IMF: Integrating Matched Features Using Attentive Logit in Knowledge Distillation IJCAI 2023

Quantization-aware and Tensor-compressed Training of Transformers for Natural Language Understanding INTERSPEECH 2023

Efficient Controllable Multi-Task Architectures ICCV 2023

TexQ: Zero-shot Network Quantization with Texture Feature Distribution Calibration NIPS 2023

UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors ICCV 2023

QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D Object Detection ICCV 2023

Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages ACL 2023