Knowledge Distillation

Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models NeurIPS 2022 Fine-Tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning CVPR 2022 Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively NeurIPS 2022 FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Models Interspeech 2022 Fixing Model Bugs with Natural Language Patches EMNLP 2022 Focal and Global Knowledge Distillation for Detectors CVPR 2022 FOCUS: Flexible Optimizable Counterfactual Explanations for Tree Ensembles AAAI 2022 FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models NAACL 2022 From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models EMNLP 2022 FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metrics for Automatic Text Generation ACL 2022 Functional Ensemble Distillation NeurIPS 2022 Generic Overgeneralization in Pre-trained Language Models COLING 2022 Geometric Knowledge Distillation: Topology Compression for Graph Neural Networks NeurIPS 2022 Graph Condensation for Graph Neural Networks ICLR 2022 Graphein - a Python Library for Geometric Deep Learning and Network Analysis on Biomolecular Structures and Interaction Networks NeurIPS 2022 Graph-less Neural Networks: Teaching Old MLPs New Tricks Via Distillation ICLR 2022 Grounding in social media: An approach to building a chit-chat dialogue model NAACL 2022 Hard Gate Knowledge Distillation - Leverage Calibration for Robust and Reliable Language Model EMNLP 2022 Harmless Transfer Learning for Item Embeddings NAACL 2022 Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning IJCAI 2022 Hierarchical Channel-spatial Encoding for Communication-efficient Collaborative Learning NeurIPS 2022 Hilbert Distillation for Cross-Dimensionality Networks NeurIPS 2022 How Can a Teacher Make Learning From Sparse Data Softer? Application to Business Relation Extraction EMNLP 2022 How Many Observations Are Enough? Knowledge Distillation for Trajectory Forecasting CVPR 2022 HW-TSC’s Submissions to the WMT 2022 General Machine Translation Shared Task EMNLP 2022