Knowledge Distillation

Adversarial Local Distribution Regularization for Knowledge Distillation WACV 2023 A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation ICCV 2023 Alleviating Catastrophic Forgetting of Incremental Object Detection via Within-Class and Between-Class Knowledge Distillation ICCV 2023 A Metric-Driven Approach to Conformer Layer Pruning for Efficient ASR Inference Interspeech 2023 A Multiple-Teacher Pruning Based Self-Distillation (MT-PSD) Approach to Model Compression for Audio-Visual Wake Word Spotting Interspeech 2023 An Efficient and Noise-Robust Audiovisual Encoder for Audiovisual Speech Recognition Interspeech 2023 An Ensemble Distillation Framework for Sentence Embeddings with Multilingual Round-Trip Translation AAAI 2023 A Neural Span-Based Continual Named Entity Recognition Model AAAI 2023 Angelic Patches for Improving Third-Party Object Detector Performance CVPR 2023 An Investigation of the Combination of Rehearsal and Knowledge Distillation in Continual Learning for Spoken Language Understanding Interspeech 2023 AnyDA: Anytime Domain Adaptation ICLR 2023 Application of Knowledge Distillation to Multi-Task Speech Representation Learning Interspeech 2023 Architecture, Dataset and Model-Scale Agnostic Data-Free Meta-Learning CVPR 2023 Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method ACL 2023 Are Large Kernels Better Teachers than Transformers for ConvNets? ICML 2023 A Scalable and Efficient Iterative Method for Copying Machine Learning Classifiers JMLR 2023 A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance ICCV 2023 A Strong Baseline for Generalized Few-Shot Semantic Segmentation CVPR 2023 A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models ACL 2023 A Study on the Efficiency and Generalization of Light Hybrid Retrievers ACL 2023 A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training ACL 2023 A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation ICLR 2023 A Teacher-Student Approach for Extracting Informative Speaker Embeddings From Speech Mixtures Interspeech 2023 Attend, Select and Eliminate: Accelerating Multi-turn Response Selection with Dual-attention-based Content Elimination ACL 2023 Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection AAAI 2023