← Learning Types
Deep Learning › Learning Types ›

Knowledge Distillation

790 directly classified papers

Papers per year

Papers

Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations ACL 2022
BERT Learns to Teach: Knowledge Distillation with Meta Learning ACL 2022
Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal ACL 2022
Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation ACL 2022
Metadata Shaping: A Simple Approach for Knowledge-Enhanced Language Models ACL 2022
Low Resource Causal Event Detection from Biomedical Literature ACL 2022
Model Distillation for Faithful Explanations of Medical Code Predictions ACL 2022
Domain-specific knowledge distillation yields smaller and better models for conversational commerce ACL 2022
Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks NIPS 2022
ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation NIPS 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time ICML 2022
Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation ICML 2022
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models NAACL 2022
Knowledge Inheritance for Pre-trained Language Models NAACL 2022
Transferring Knowledge from Structure-aware Self-attention Language Model to Sequence-to-Sequence Semantic Parsing COLING 2022
Co-Segmentation Aided Two-Stream Architecture for Video Captioning WACV 2022
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection EMNLP 2022
Biomedical NER for the Enterprise with Distillated BERN2 and the Kazu Framework EMNLP 2022
Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval EMNLP 2022
XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding ACL 2022
Adversarial Knowledge Distillation For Robust Spoken Language Understanding INTERSPEECH 2022
Knowledge Distillation For CTC-based Speech Recognition Via Consistent Acoustic Representation Learning INTERSPEECH 2022
Data-augmented cross-lingual synthesis in a teacher-student framework INTERSPEECH 2022
Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Models INTERSPEECH 2022
Target-Relevant Knowledge Preservation for Multi-Source Domain Adaptive Object Detection CVPR 2022