Knowledge Distillation

PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization CVPR 2024 Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners CVPR 2024 Privileged Prior Information Distillation for Image Matting AAAI 2024 Probabilistic Conformal Distillation for Enhancing Missing Modality Robustness NeurIPS 2024 Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion AAAI 2024 Progressively Knowledge Distillation via Re-parameterizing Diffusion Reverse Process AAAI 2024 PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA ACL 2024 Prompting Large Language Models with Audio for General-Purpose Speech Summarization Interspeech 2024 PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning EMNLP 2024 PromptKD: Unsupervised Prompt Distillation for Vision-Language Models CVPR 2024 Prophecy Distillation for Boosting Abstractive Summarization COLING 2024 ProS: Facial Omni-Representation Learning via Prototype-Based Self-Distillation WACV 2024 PRVAE-VC2: Non-Parallel Voice Conversion by Distillation of Speech Representations Interspeech 2024 Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning ICLR 2024 QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models ICLR 2024 QBB: Quantization with Binary Bases for LLMs NeurIPS 2024 Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning NAACL 2024 Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models ACL 2024 Querying as Prompt: Parameter-Efficient Learning for Multimodal Language Model CVPR 2024 Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How ICLR 2024 RA2FD: Distilling Faithfulness into Efficient Dialogue Systems EMNLP 2024 RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Features CVPR 2024 RADCoT: Retrieval-Augmented Distillation to Specialization Models for Generating Chain-of-Thoughts in Query Expansion COLING 2024 RaD-Net 2: A causal two-stage repairing and denoising speech enhancement network with knowledge distillation and complex axial self-attention Interspeech 2024 RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation AAAI 2024