Knowledge Distillation

Self-Distilled Depth Refinement with Noisy Poisson Fusion NeurIPS 2024 ShareBERT: Embeddings Are Capable of Learning Hidden Layers AAAI 2024 Simple and Fast Distillation of Diffusion Models NeurIPS 2024 SinSR: Diffusion-Based Image Super-Resolution in a Single Step CVPR 2024 SlimSAM: 0.1% Data Makes Segment Anything Slim NeurIPS 2024 Small But Funny: A Feedback-Driven Approach to Humor Distillation ACL 2024 Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs ACL 2024 Small Scale Data-Free Knowledge Distillation CVPR 2024 Speedy Gonzales: A Collection of Fast Task-Specific Models for Spanish NAACL 2024 SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams NeurIPS 2024 SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation AAAI 2024 Structured Unrestricted-Rank Matrices for Parameter Efficient Finetuning NeurIPS 2024 Study Selectively: An Adaptive Knowledge Distillation based on a Voting Network for Heart Sound Classification Interspeech 2024 StyleGAN-Fusion: Diffusion Guided Domain Adaptation of Image Generators WACV 2024 StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements EMNLP 2024 SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation EMNLP 2024 Task-Customized Mixture of Adapters for General Image Fusion CVPR 2024 Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios ACL 2024 Teaching Small Language Models Reasoning through Counterfactual Distillation EMNLP 2024 Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering ACL 2024 Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024 Text-Enhanced Data-free Approach for Federated Class-Incremental Learning CVPR 2024 The Mamba in the Llama: Distilling and Accelerating Hybrid Models NeurIPS 2024 Three Pillars Improving Vision Foundation Model Distillation for Lidar CVPR 2024 Time- Memory- and Parameter-Efficient Visual Adaptation CVPR 2024