conftrace_

student model

107 papers

Explore in graph

Co-occurring keywords

knowledge distillation (3725) model compression (3302) teacher model (53) large language model (13587) language model (4599) transfer learning (5449) neural network (6616) representation learning (6206) text classification (6793) relational knowledge (26)

Papers

SMASH: Improving SMAll Language Models’ Few-SHot Ability with Prompt-Based Distillation EMNLP 2022
Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network EMNLP 2022
Knowledge Distillation As Efficient Pre-Training: Faster Convergence, Higher Data-Efficiency, and Better Transferability CVPR 2022
Learning to Scaffold: Optimizing Model Explanations for Teaching NIPS 2022
Multi-Granularity Structural Knowledge Distillation for Language Model Compression ACL 2022
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation NAACL 2022
Rethinking Task-Specific Knowledge Distillation: Contextualized Corpus as Better Textbook EMNLP 2022
COACH: Cooperative Robot Teaching CORL 2022
Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks EMNLP 2022
Respecting Transfer Gap in Knowledge Distillation NIPS 2022
Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models NAACL 2022
KNOT: Knowledge Distillation Using Optimal Transport for Solving NLP Tasks COLING 2022
Sparse Teachers Can Be Dense with Knowledge EMNLP 2022
Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains ACL 2021
Learning Light-Weight Translation Models from Deep Transformer AAAI 2021
Multi-Task Self-Training for Learning General Representations ICCV 2021
RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation EMNLP 2021
KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation IJCAI 2021
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation IJCNLP 2021
Noise as a Resource for Learning in Knowledge Distillation WACV 2021
Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task EMNLP 2020
Revisiting Knowledge Distillation via Label Smoothing Regularization CVPR 2020
Online Knowledge Distillation with Diverse Peers AAAI 2020
Search to Distill: Pearls Are Everywhere but Not the Eyes CVPR 2020
Distilling Knowledge From Graph Convolutional Networks CVPR 2020