← Application Areas
Machine Learning › Application Areas ›

Knowledge Distillation

2907 directly classified papers

Papers per year

Papers

RD-Suite: A Benchmark for Ranking Distillation NIPS 2023
Cluster-aware Semi-supervised Learning: Relational Knowledge Distillation Provably Learns Clustering NIPS 2023
On the Size and Approximation Error of Distilled Datasets NIPS 2023
KD-Zero: Evolving Knowledge Distiller for Any Teacher-Student Pairs NIPS 2023
Module-wise Adaptive Distillation for Multimodality Foundation Models NIPS 2023
Accelerating Molecular Graph Neural Networks via Knowledge Distillation NIPS 2023
Foundation Model is Efficient Multimodal Multitask Model Selector NIPS 2023
NEO-KD: Knowledge-Distillation-Based Adversarial Training for Robust Multi-Exit Neural Networks NIPS 2023
Multi-Head Adapter Routing for Cross-Task Generalization NIPS 2023
FedFed: Feature Distillation against Data Heterogeneity in Federated Learning NIPS 2023
Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models NIPS 2023
Metis: Understanding and Enhancing In-Network Regular Expressions NIPS 2023
Should Under-parameterized Student Networks Copy or Average Teacher Weights? NIPS 2023
Fair Graph Distillation NIPS 2023
Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models NIPS 2023
Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks NIPS 2023
Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer NIPS 2023
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions NIPS 2023
Lifelong Compression Mixture Model via Knowledge Relationship Graph AAAI 2023
CrysGNN: Distilling Pre-trained Knowledge to Enhance Property Prediction for Crystalline Materials AAAI 2023
LIMIP: Lifelong Learning to Solve Mixed Integer Programs AAAI 2023
Darwinian Model Upgrades: Model Evolving with Selective Compatibility AAAI 2023
Efficient Image Captioning for Edge Devices AAAI 2023
SKDBERT: Compressing BERT via Stochastic Knowledge Distillation AAAI 2023
Adaptive Hierarchy-Branch Fusion for Online Knowledge Distillation AAAI 2023