Knowledge Distillation

BabyLM Challenge: Experimenting with Self-Distillation and Reverse-Distillation for Language Model Pre-Training on Constrained Datasets CoNLL 2024 Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection AAAI 2024 Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model ACL 2024 BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts NeurIPS 2024 Batched Low-Rank Adaptation of Foundation Models ICLR 2024 Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information ICLR 2024 Benchmarking Diffusion Models for Machine Translation EACL 2024 Better Alignment with Instruction Back-and-Forth Translation EMNLP 2024 Beyond Full Fine-tuning: Harnessing the Power of LoRA for Multi-Task Instruction Tuning COLING 2024 BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation ACL 2024 BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling NeurIPS 2024 Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters CVPR 2024 Boosting Residual Networks with Group Knowledge AAAI 2024 Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models? EMNLP 2024 Boosting Self-Supervision for Single-View Scene Completion via Knowledge Distillation CVPR 2024 Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Models CVPR 2024 BPKD: Boundary Privileged Knowledge Distillation for Semantic Segmentation WACV 2024 Breaking Barriers of System Heterogeneity: Straggler-Tolerant Multimodal Federated Learning via Knowledge Distillation IJCAI 2024 Bridging Language Gaps in Audio-Text Retrieval Interspeech 2024 BSNet: Box-Supervised Simulation-assisted Mean Teacher for 3D Instance Segmentation CVPR 2024 BS-PLCNet 2: Two-stage Band-split Packet Loss Concealment Network with Intra-model Knowledge Distillation Interspeech 2024 Building Vision-Language Models on Solid Foundations with Masked Distillation CVPR 2024 C2KD: Bridging the Modality Gap for Cross-Modal Knowledge Distillation CVPR 2024 Cache & Distil: Optimising API Calls to Large Language Models ACL 2024 CaKDP: Category-aware Knowledge Distillation and Pruning Framework for Lightweight 3D Object Detection CVPR 2024