knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Dual Expert Distillation Network for Generalized Zero-Shot Learning IJCAI 2024

FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning AAAI 2024

Decoupled Kullback-Leibler Divergence Loss NIPS 2024

Gradient Reweighting: Towards Imbalanced Class-Incremental Learning CVPR 2024

A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition CVPR 2024

Efficient Speaker Embedding Extraction Using a Twofold Sliding Window Algorithm for Speaker Diarization INTERSPEECH 2024

Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Study ACL 2024

Exploring compressibility of transformer based text-to-music (TTM) models INTERSPEECH 2024

Compact Speech Translation Models via Discrete Speech Units Pretraining ACL 2024

CTC-based Non-autoregressive Textless Speech-to-Speech Translation ACL 2024

Speedy Gonzales: A Collection of Fast Task-Specific Models for Spanish NAACL 2024

UniADS: Universal Architecture-Distiller Search for Distillation Gap AAAI 2024

Federated Learning with Extremely Noisy Clients via Negative Distillation AAAI 2024

Learn to Rectify the Bias of CLIP for Unsupervised Semantic Segmentation CVPR 2024

Large Language Models for Data Annotation and Synthesis: A Survey EMNLP 2024

AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation EMNLP 2024

tinyCLAP: Distilling Constrastive Language-Audio Pretrained Models INTERSPEECH 2024

Study Selectively: An Adaptive Knowledge Distillation based on a Voting Network for Heart Sound Classification INTERSPEECH 2024

DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models EMNLP 2024

TinyChart: Efficient Chart Understanding with Program-of-Thoughts Learning and Visual Token Merging EMNLP 2024

Unsupervised Human Preference Learning EMNLP 2024

Performance-Guided LLM Knowledge Distillation for Efficient Text Classification at Scale EMNLP 2024

Advancing Large Language Model Attribution through Self-Improving EMNLP 2024

AlignCap: Aligning Speech Emotion Captioning to Human Preferences EMNLP 2024

Teaching Small Language Models Reasoning through Counterfactual Distillation EMNLP 2024