← Techniques
Deep Learning › Techniques ›

Knowledge Distillation

623 directly classified papers

Papers per year

Papers

Towards Higher Pareto Frontier in Multilingual Machine Translation ACL 2023
Less is More: Task-aware Layer-wise Distillation for Language Model Compression ICML 2023
MobileVOS: Real-Time Video Object Segmentation Contrastive Learning Meets Knowledge Distillation CVPR 2023
Connective Prediction for Implicit Discourse Relation Recognition via Knowledge Distillation ACL 2023
mCLIP: Multilingual CLIP via Cross-lingual Transfer ACL 2023
Plug-and-Play Document Modules for Pre-trained Models ACL 2023
Distilling Reasoning Capabilities into Smaller Language Models ACL 2023
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer ACL 2023
DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translation ACL 2023
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference ACL 2023
Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction ACL 2023
Better Language Models of Code through Self-Improvement ACL 2023
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning ACL 2023
Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints ACL 2023
Are Large Kernels Better Teachers than Transformers for ConvNets? ICML 2023
On the Impact of Knowledge Distillation for Model Interpretability ICML 2023
Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning ICML 2023
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers EACL 2023
StructVPR: Distill Structural Knowledge With Weighting Samples for Visual Place Recognition CVPR 2023
Heterogeneous-Branch Collaborative Learning for Dialogue Generation AAAI 2023
Quantized Feature Distillation for Network Quantization AAAI 2023
DistilPose: Tokenized Pose Regression With Heatmap Distillation CVPR 2023
Enhanced Multimodal Representation Learning With Cross-Modal KD CVPR 2023
An Ensemble Distillation Framework for Sentence Embeddings with Multilingual Round-Trip Translation AAAI 2023
Cross-Modal Distillation for Speaker Recognition AAAI 2023