Knowledge Distillation

Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection CVPR 2023 Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks NeurIPS 2023 Learn From One Specialized Sub-Teacher: One-to-One Mapping for Feature-Based Knowledge Distillation EMNLP 2023 Learning Easily Updated General Purpose Text Representations with Adaptable Task-Specific Prefix EMNLP 2023 Learning from a Friend: Improving Event Extraction via Self-Training with Feedback from Abstract Meaning Representation ACL 2023 LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation CVPR 2023 Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models EMNLP 2023 Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference ACL 2023 New Insights for the Stability-Plasticity Dilemma in Online Continual Learning ICLR 2023 Scaffolding a Student to Instill Knowledge ICLR 2023 SLaM: Student-Label Mixing for Distillation with Unlabeled Examples NeurIPS 2023 StructVPR: Distill Structural Knowledge With Weighting Samples for Visual Place Recognition CVPR 2023 STXD: Structural and Temporal Cross-Modal Distillation for Multi-View 3D Object Detection NeurIPS 2023 Teacher Guided Training: An Efficient Framework for Knowledge Transfer ICLR 2023 Watermarking PLMs on Classification Tasks by Combining Contrastive Learning with Weight Perturbation EMNLP 2023 BERT Learns to Teach: Knowledge Distillation with Meta Learning ACL 2022 Calibrating Student Models for Emotion-related Tasks EMNLP 2022 Distilling Multilingual Transformers into CNNs for Scalable Intent Classification EMNLP 2022 HyperInverter: Improving StyleGAN Inversion via Hypernetwork CVPR 2022 Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks EMNLP 2022 Knowledge Distillation via the Target-Aware Transformer CVPR 2022 Knowledge Transfer with Visual Prompt in multi-modal Dialogue Understanding and Generation COLING 2022 Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation EMNLP 2022 Multi-Granularity Structural Knowledge Distillation for Language Model Compression ACL 2022 Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search CVPR 2022