← Techniques
Deep Learning › Techniques ›

Knowledge Distillation

623 directly classified papers

Papers per year

Papers

Speedy Gonzales: A Collection of Fast Task-Specific Models for Spanish NAACL 2024
PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA ACL 2024
Small Scale Data-Free Knowledge Distillation CVPR 2024
To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation ACL 2024
P2D: Plug and Play Discriminator for Accelerating GAN Frameworks WACV 2024
Distilling Semantic Priors from SAM to Efficient Image Restoration Models CVPR 2024
Towards Continual Knowledge Graph Embedding via Incremental Distillation AAAI 2024
SARCAT: Generative Span-Act Guided Response Generation using Copy-enhanced Target Augmentation EMNLP 2024
Efficient Dataset Distillation via Minimax Diffusion CVPR 2024
KG-Adapter: Enabling Knowledge Graph Integration in Large Language Models through Parameter-Efficient Fine-Tuning ACL 2024
Contextual Affinity Distillation for Image Anomaly Detection WACV 2024
$\texttt{Model-GLUE}$: Democratized LLM Scaling for A Large Model Zoo in the Wild NIPS 2024
UDON: Universal Dynamic Online distillatioN for generic image representations NIPS 2024
LoRA Meets Dropout under a Unified Framework ACL 2024
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models EMNLP 2024
Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints ACL 2023
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning ACL 2023
Better Language Models of Code through Self-Improvement ACL 2023
Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction ACL 2023
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference ACL 2023
DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translation ACL 2023
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer ACL 2023
Distilling Reasoning Capabilities into Smaller Language Models ACL 2023
Plug-and-Play Document Modules for Pre-trained Models ACL 2023
mCLIP: Multilingual CLIP via Cross-lingual Transfer ACL 2023