Knowledge Distillation

Teacher and Student Models of Offensive Language in Social Media ACL 2023 Teacher Assistant-Based Knowledge Distillation Extracting Multi-level Features on Single Channel Sleep EEG IJCAI 2023 Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers EACL 2023 Teaching Small Language Models to Reason ACL 2023 Teaching What You Should Teach: A Data-Based Distillation Method IJCAI 2023 TempLM: Distilling Language Models into Template-Based Generators ACL 2023 TexQ: Zero-shot Network Quantization with Texture Feature Distribution Calibration NeurIPS 2023 Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation EACL 2023 Text Representation Distillation via Information Bottleneck Principle EMNLP 2023 The Augmented Image Prior: Distilling 1000 Classes by Extrapolating from a Single Image ICLR 2023 The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation ICLR 2023 The economic trade-offs of large language models: A case study ACL 2023 The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation ICLR 2023 The student becomes the master: Outperforming GPT3 on Scientific Factual Error Correction EMNLP 2023 The Unreasonable Effectiveness of Large Language-Vision Models for Source-Free Video Domain Adaptation ICCV 2023 TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance ICCV 2023 TinyHD: Efficient Video Saliency Prediction With Heterogeneous Decoders Using Hierarchical Maps Distillation WACV 2023 TinyMIM: An Empirical Study of Distilling MIM Pre-Trained Models CVPR 2023 Tiny Updater: Towards Efficient Neural Network-Driven Software Updating ICCV 2023 Token-Scaled Logit Distillation for Ternary Weight Generative Language Models NeurIPS 2023 torchdistill Meets Hugging Face Libraries for Reproducible, Coding-Free Deep Learning Studies: A Case Study on NLP EMNLP 2023 Towards a Smaller Student: Capacity Dynamic Distillation for Efficient Image Retrieval CVPR 2023 Towards a Unified Conversational Recommendation System: Multi-task Learning via Contextualized Knowledge Distillation EMNLP 2023 Towards Better Entity Linking with Multi-View Enhanced Distillation ACL 2023 Towards Efficient Pre-Trained Language Model via Feature Correlation Distillation NeurIPS 2023