Knowledge Distillation

Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Models Interspeech 2022 DENSE: Data-Free One-Shot Federated Learning NeurIPS 2022 Device-Directed Speech Detection: Regularization via Distillation for Weakly-Supervised Models Interspeech 2022 DINE: Domain Adaptation From Single and Multiple Black-Box Predictors CVPR 2022 DiPCAN: Distilling Privileged Information for Crowd-Aware Navigation RSS 2022 Discovering and Overcoming Limitations of Noise-engineered Data-free Knowledge Distillation NeurIPS 2022 Distillation of RL Policies with Formal Guarantees via Variational Abstraction of Markov Decision Processes AAAI 2022 Distillation-Resistant Watermarking for Model Protection in NLP EMNLP 2022 Distillation Using Oracle Queries for Transformer-Based Human-Object Interaction Detection CVPR 2022 Distilled Dual-Encoder Model for Vision-Language Understanding EMNLP 2022 Distilling a Pretrained Language Model to a Multilingual ASR Model Interspeech 2022 Distilling GANs with Style-Mixed Triplets for X2I Translation with Limited Data ICLR 2022 Distilling Inter-Class Distance for Semantic Segmentation IJCAI 2022 Distilling Multilingual Transformers into CNNs for Scalable Intent Classification EMNLP 2022 Distilling Representations from GAN Generator via Squeeze and Span NeurIPS 2022 Distill The Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation EMNLP 2022 Diversifying Neural Dialogue Generation via Negative Distillation NAACL 2022 DKM: Differentiable k-Means Clustering Layer for Neural Network Compression ICLR 2022 DNA: Proximal Policy Optimization with a Dual Network Architecture NeurIPS 2022 DocInfer: Document-level Natural Language Inference using Optimal Evidence Selection EMNLP 2022 Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation ACL 2022 Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation ACL 2022 Domain-specific knowledge distillation yields smaller and better models for conversational commerce ACL 2022 Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach ACL 2022 DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization ACL 2022