Knowledge Distillation

Less is More: Task-aware Layer-wise Distillation for Language Model Compression ICML 2023 Leveraging Contrastive Learning and Knowledge Distillation for Incomplete Modality Rumor Detection EMNLP 2023 Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection NeurIPS 2023 LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation CVPR 2023 Lifting the Curse of Capacity Gap in Distilling Language Models ACL 2023 LightFormer: Light-weight Transformer Using SVD-based Weight Transfer and Parameter Sharing ACL 2023 LLM aided semi-supervision for efficient Extractive Dialog Summarization EMNLP 2023 Local 3D Editing via 3D Distillation of CLIP Knowledge CVPR 2023 Long-Tailed Question Answering in an Open World ACL 2023 Masked Autoencoders Enable Efficient Knowledge Distillers CVPR 2023 Masked Video Distillation: Rethinking Masked Feature Modeling for Self-Supervised Video Representation Learning CVPR 2023 mCLIP: Multilingual CLIP via Cross-lingual Transfer ACL 2023 MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition ICCV 2023 MEDIC: Remove Model Backdoors via Importance Driven Cloning CVPR 2023 MEID: Mixture-of-Experts with Internal Distillation for Long-Tailed Video Recognition AAAI 2023 Mixture Uniform Distribution Modeling and Asymmetric Mix Distillation for Class Incremental Learning AAAI 2023 MobileVOS: Real-Time Video Object Segmentation Contrastive Learning Meets Knowledge Distillation CVPR 2023 Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference ACL 2023 Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning CVPR 2023 NEO-KD: Knowledge-Distillation-Based Adversarial Training for Robust Multi-Exit Neural Networks NeurIPS 2023 Neural TSP Solver with Progressive Distillation AAAI 2023 NIFF: Alleviating Forgetting in Generalized Few-Shot Object Detection via Neural Instance Feature Forging CVPR 2023 Object-Aware Distillation Pyramid for Open-Vocabulary Object Detection CVPR 2023 On Distillation of Guided Diffusion Models CVPR 2023 One Is All: Bridging the Gap between Neural Radiance Fields Architectures with Progressive Volume Distillation AAAI 2023