Knowledge Distillation

Accelerating Inference for Pretrained Language Models by Unified Multi-Perspective Early Exiting COLING 2022 Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection ACL 2022 A Closer Look at Parameter Contributions When Training Neural Language and Translation Models COLING 2022 ADD: Frequency Attention and Multi-View Based Knowledge Distillation to Detect Low-Quality Compressed Deepfake Images AAAI 2022 Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers AAAI 2022 Adversarial Knowledge Distillation For Robust Spoken Language Understanding Interspeech 2022 A Flexible Multi-Task Model for BERT Serving ACL 2022 Aligned Weight Regularizers for Pruning Pretrained Neural Networks ACL 2022 An Effective, Performant Named Entity Recognition System for Noisy Business Telephone Conversation Transcripts COLING 2022 An Efficient Active Learning Pipeline for Legal Text Classification EMNLP 2022 An Efficient Combinatorial Optimization Model Using Learning-to-Rank Distillation AAAI 2022 Anomaly Detection via Reverse Distillation From One-Class Embedding CVPR 2022 An Unsupervised Multiple-Task and Multiple-Teacher Model for Cross-lingual Named Entity Recognition ACL 2022 A Scalable Model Specialization Framework for Training and Inference using Submodels and its Application to Speech Model Personalization Interspeech 2022 A Self-improving Skin Lesions Diagnosis Framework Via Pseudo-labeling and Self-distillation ACML 2022 Asymmetric Temperature Scaling Makes Larger Networks Teach Well Again NeurIPS 2022 Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks NAACL 2022 Attention Temperature Matters in Abstractive Summarization Distillation ACL 2022 Attribute Injection for Pretrained Language Models: A New Benchmark and an Efficient Method COLING 2022 Augmented Geometric Distillation for Data-Free Incremental Person ReID CVPR 2022 Bag of Instances Aggregation Boosts Self-supervised Distillation ICLR 2022 Being Friends Instead of Adversaries: Deep Networks Learn from Data Simplified by Other Networks AAAI 2022 BEiT: BERT Pre-Training of Image Transformers ICLR 2022 Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms NeurIPS 2022 bert2BERT: Towards Reusable Pretrained Language Models ACL 2022