knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

CoIN: A Benchmark of Continual Instruction Tuning for Multimodel Large Language Models NIPS 2024

2D Feature Distillation for Weakly- and Semi-Supervised 3D Semantic Segmentation WACV 2024

LP-OVOD: Open-Vocabulary Object Detection by Linear Probing WACV 2024

Cross-Feature Contrastive Loss for Decentralized Deep Learning on Heterogeneous Data WACV 2024

Multi-View Classification Using Hybrid Fusion and Mutual Distillation WACV 2024

LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis WACV 2024

Towards Domain-Aware Knowledge Distillation for Continual Model Generalization WACV 2024

Kaizen: Practical Self-Supervised Continual Learning With Continual Fine-Tuning WACV 2024

Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering ACL 2024

DistillMIKE: Editing Distillation of Massive In-Context Knowledge Editing in Large Language Models ACL 2024

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning ACL 2024

FAKD: Feature Augmented Knowledge Distillation for Semantic Segmentation WACV 2024

Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages ACL 2024

Continual Learning with Semi-supervised Contrastive Distillation for Incremental Neural Machine Translation ACL 2024

FastAST: Accelerating Audio Spectrogram Transformer via Token Merging and Cross-Model Knowledge Distillation INTERSPEECH 2024

VkD: Improving Knowledge Distillation using Orthogonal Projections CVPR 2024

CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation CVPR 2024

MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric CVPR 2024

Asymmetric Masked Distillation for Pre-Training Small Foundation Models CVPR 2024

SelfCodeAlign: Self-Alignment for Code Generation NIPS 2024

Efficient Multi-task LLM Quantization and Serving for Multiple LoRA Adapters NIPS 2024

Federated Model Heterogeneous Matryoshka Representation Learning NIPS 2024

Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE NIPS 2024

OD-DETR: Online Distillation for Stabilizing Training of Detection Transformer IJCAI 2024

SHAP-EDITOR: Instruction-Guided Latent 3D Editing in Seconds CVPR 2024