Knowledge Distillation

UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition ICLR 2024 Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decoding ACL 2024 Unsupervised Human Preference Learning EMNLP 2024 Unveiling the Magic: Investigating Attention Distillation in Retrieval-Augmented Generation NAACL 2024 Updating Large Language Models’ Memories with Time Constraints EMNLP 2024 Using Early Readouts To Mediate Featural Bias in Distillation WACV 2024 VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks NeurIPS 2024 VE-KD: Vocabulary-Expansion Knowledge-Distillation for Training Smaller Domain-Specific Language Models EMNLP 2024 VerifiNER: Verification-augmented NER via Knowledge-grounded Reasoning with Large Language Models ACL 2024 VeXKD: The Versatile Integration of Cross-Modal Fusion and Knowledge Distillation for 3D Perception NeurIPS 2024 Video Event Extraction with Multi-View Interaction Knowledge Distillation AAAI 2024 Visual Editing with LLM-based Tool Chaining: An Efficient Distillation Approach for Real-Time Applications EMNLP 2024 Visual Fourier Prompt Tuning NeurIPS 2024 Visual Perception by Large Language Model’s Weights NeurIPS 2024 Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models CVPR 2024 VkD: Improving Knowledge Distillation using Orthogonal Projections CVPR 2024 Voxel Proposal Network via Multi-Frame Knowledge Distillation for Semantic Scene Completion NeurIPS 2024 VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs ICLR 2024 Wakening Past Concepts Without Past Data: Class-Incremental Learning From Online Placebos WACV 2024 Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation NeurIPS 2024 Weak-to-Strong 3D Object Detection with X-Ray Distillation CVPR 2024 WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark NeurIPS 2024 Weight-Inherited Distillation for Task-Agnostic BERT Compression NAACL 2024 What Sketch Explainability Really Means for Downstream Tasks? CVPR 2024 When Babies Teach Babies: Can student knowledge sharing outperform Teacher-Guided Distillation on small datasets? CoNLL 2024