Knowledge Distillation

CALICO: Self-Supervised Camera-LiDAR Contrastive Pre-training for BEV Perception ICLR 2024 CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning ACL 2024 Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study NeurIPS 2024 Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs ACL 2024 CapEEN: Image Captioning with Early Exits and Knowledge Distillation EMNLP 2024 CAR-Transformer: Cross-Attention Reinforcement Transformer for Cross-Lingual Summarization AAAI 2024 Centrality-aware Product Retrieval and Ranking EMNLP 2024 Change Is the Only Constant: Dynamic LLM Slicing based on Layer Redundancy EMNLP 2024 Choosy Babies Need One Coach: Inducing Mode-Seeking Behavior in BabyLlama with Reverse KL Divergence CoNLL 2024 CIFD: Controlled Information Flow to Enhance Knowledge Distillation NeurIPS 2024 CiteME: Can Language Models Accurately Cite Scientific Claims? NeurIPS 2024 Class-Incremental Few-Shot Event Detection COLING 2024 Class Incremental Learning with Multi-Teacher Distillation CVPR 2024 CLIP-KD: An Empirical Study of CLIP Model Distillation CVPR 2024 CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Prediction ICLR 2024 Cloud-Device Collaborative Learning for Multimodal Large Language Models CVPR 2024 Cloud Object Detector Adaptation by Integrating Different Source Knowledge NeurIPS 2024 Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models ACL 2024 CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation CVPR 2024 Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment ACL 2024 COMBHelper: A Neural Approach to Reduce Search Space for Graph Combinatorial Problems AAAI 2024 Compact Language Models via Pruning and Knowledge Distillation NeurIPS 2024 Complementary Knowledge Distillation for Robust and Privacy-Preserving Model Serving in Vertical Federated Learning AAAI 2024 Confidence Matters: Enhancing Medical Image Classification Through Uncertainty-Driven Contrastive Self-Distillation MICCAI 2024 CONSCENDI: A Contrastive and Scenario-Guided Distillation Approach to Guardrail Models for Virtual Assistants NAACL 2024