knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Speaker Change Detection with Weighted-sum Knowledge Distillation based on Self-supervised Pre-trained Models INTERSPEECH 2024

Contextualization Distillation from Large Language Model for Knowledge Graph Completion EACL 2024

Pre-Training Methods for Question Reranking EACL 2024

Small Language Models Improve Giants by Rewriting Their Outputs EACL 2024

Learning to Retrieve In-Context Examples for Large Language Models EACL 2024

BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation ACL 2024

CLIP-Guided Federated Learning on Heterogeneity and Long-Tailed Data AAAI 2024

Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models IJCAI 2024

BabyHGRN: Exploring RNNs for Sample-Efficient Language Modeling CONLL 2024

BS-PLCNet 2: Two-stage Band-split Packet Loss Concealment Network with Intra-model Knowledge Distillation INTERSPEECH 2024

Frequency-mix Knowledge Distillation for Fake Speech Detection INTERSPEECH 2024

FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning IJCAI 2024

Resource-Efficient Transformer Pruning for Finetuning of Large Models CVPR 2024

Unlocking Pre-trained Image Backbones for Semantic Image Synthesis CVPR 2024

Ranking Distillation for Open-Ended Video Question Answering with Insufficient Labels CVPR 2024

Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward IJCAI 2024

Knowledge Distillation in Federated Learning: A Practical Guide IJCAI 2024

Generative Model-Based Feature Knowledge Distillation for Action Recognition AAAI 2024

Efficient Audio Captioning with Encoder-Level Knowledge Distillation INTERSPEECH 2024

AQ-DETR: Low-Bit Quantized Detection Transformer with Auxiliary Queries AAAI 2024

Distilling Reliable Knowledge for Instance-Dependent Partial Label Learning AAAI 2024

Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization AAAI 2024

Mitigating Overfitting in Structured Pruning of ASR Models with Gradient-Guided Parameter Regularization INTERSPEECH 2024

Self-Training Based Few-Shot Node Classification by Knowledge Distillation AAAI 2024

Vec-Tok-VC+: Residual-enhanced Robust Zero-shot Voice Conversion with Progressive Constraints in a Dual-mode Training Strategy INTERSPEECH 2024