Knowledge Distillation

AM-RADIO: Agglomerative Vision Foundation Model Reduce All Domains Into One CVPR 2024 An Empirical Study of Multilingual Reasoning Distillation for Question Answering EMNLP 2024 A Privacy-preserving Approach to Ingest Knowledge from Proprietary Web-based to Locally Run Models for Medical Progress Note Generation ACL 2024 AS-ES Learning: Towards efficient CoT learning in small models ACL 2024 Asymmetric Masked Distillation for Pre-Training Small Foundation Models CVPR 2024 BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts NeurIPS 2024 Bellman Optimal Stepsize Straightening of Flow-Matching Models ICLR 2024 BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation ACL 2024 Bi-ViT: Pushing the Limit of Vision Transformer Quantization AAAI 2024 Boosting Residual Networks with Group Knowledge AAAI 2024 Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models? EMNLP 2024 Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Models CVPR 2024 Bootstrapping Cognitive Agents with a Large Language Model AAAI 2024 Bootstrapping SparseFormers from Vision Foundation Models CVPR 2024 Building Variable-Sized Models via Learngene Pool AAAI 2024 Can Graph Neural Networks Expose Training Data Properties? An Efficient Risk Assessment Approach NeurIPS 2024 Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study NeurIPS 2024 Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models with Personality-Indicative Data EMNLP 2024 Class Incremental Learning with Multi-Teacher Distillation CVPR 2024 CLIP-Guided Federated Learning on Heterogeneity and Long-Tailed Data AAAI 2024 CLIP-KD: An Empirical Study of CLIP Model Distillation CVPR 2024 Cloud Object Detector Adaptation by Integrating Different Source Knowledge NeurIPS 2024 Cluster-Learngene: Inheriting Adaptive Clusters for Vision Transformers NeurIPS 2024 Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models ACL 2024 CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation CVPR 2024