student model

SMASH: Improving SMAll Language Models’ Few-SHot Ability with Prompt-Based Distillation EMNLP 2022 Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models NAACL 2022 Sparse Teachers Can Be Dense with Knowledge EMNLP 2022 Teach Less, Learn More: On the Undistillable Classes in Knowledge Distillation NeurIPS 2022 Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network EMNLP 2022 KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation IJCAI 2021 Learning Light-Weight Translation Models from Deep Transformer AAAI 2021 Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains ACL 2021 MixKD: Towards Efficient Distillation of Large-scale Language Models ICLR 2021 Multi-Task Self-Training for Learning General Representations ICCV 2021 Noise as a Resource for Learning in Knowledge Distillation WACV 2021 RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation EMNLP 2021 Distilling Knowledge From Graph Convolutional Networks CVPR 2020 Online Knowledge Distillation with Diverse Peers AAAI 2020 Revisiting Knowledge Distillation via Label Smoothing Regularization CVPR 2020 Search to Distill: Pearls Are Everywhere but Not the Eyes CVPR 2020 Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task EMNLP 2020 Towards Oracle Knowledge Distillation with Neural Architecture Search AAAI 2020 Amalgamating Knowledge towards Comprehensive Classification AAAI 2019 Distilling Object Detectors With Fine-Grained Feature Imitation CVPR 2019 Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning IJCAI 2019 Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing AAAI 2019 Online Model Distillation for Efficient Video Inference ICCV 2019 Relational Knowledge Distillation CVPR 2019 Scalable Private Learning with PATE ICLR 2018