Knowledge Distillation

PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization CVPR 2024 Privileged Prior Information Distillation for Image Matting AAAI 2024 Probabilistic Conformal Distillation for Enhancing Missing Modality Robustness NeurIPS 2024 Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion AAAI 2024 Progressively Knowledge Distillation via Re-parameterizing Diffusion Reverse Process AAAI 2024 PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA ACL 2024 PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning EMNLP 2024 Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models ACL 2024 RA2FD: Distilling Faithfulness into Efficient Dialogue Systems EMNLP 2024 Ranking Distillation for Open-Ended Video Question Answering with Insufficient Labels CVPR 2024 RDRec: Rationale Distillation for LLM-based Recommendation ACL 2024 Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing EMNLP 2024 Reducing Spatial Fitting Error in Distillation of Denoising Diffusion Models AAAI 2024 Reinforced Cross-Domain Knowledge Distillation on Time Series Data NeurIPS 2024 Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract) AAAI 2024 Rethinking Reverse Distillation for Multi-Modal Anomaly Detection AAAI 2024 Reviewing the Forgotten Classes for Domain Adaptation of Black-Box Predictors AAAI 2024 RRADistill: Distilling LLMs’ Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Engine EMNLP 2024 SAM-Guided Masked Token Prediction for 3D Scene Understanding NeurIPS 2024 SARCAT: Generative Span-Act Guided Response Generation using Copy-enhanced Target Augmentation EMNLP 2024 ScaleKD: Strong Vision Transformers Could Be Excellent Teachers NeurIPS 2024 SDPose: Tokenized Pose Estimation via Circulation-Guide Self-Distillation CVPR 2024 SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking CVPR 2024 Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages EMNLP 2024 Self-Distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach AAAI 2024