DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks

Jie Fu; Hongyin Luo; Jiashi Feng; Kian Hsiang Low; Tat-Seng Chua

2016 IJCAI IJCAI 2016

DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks

📈 Trend Setter - Knowledge Distillation

🐣 Hot Topic Early Bird - knowledge distillation

🐝 Cross-Pollinator - Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Speech & Audio

Authors

Jie Fu , Hongyin Luo , Jiashi Feng , Kian Hsiang Low , Tat-Seng Chua

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Application Areas > Knowledge Distillation

Keywords

hyperparameter optimization knowledge distillation automatic differentiation

Download PDF

Related papers

Unsupervised Feature Learning from Time Series 2016

Neural Generative Question Answering 2016

∃-STRIPS: Existential Quantification in Planning and Constraint Satisfaction 2016

Unsupervised Learning on Neural Network Outputs: With Application in Zero-Shot Learning 2016

Belief Update for Proper Epistemic Knowledge Bases 2016