MADA: Meta-Adaptive Optimizers Through Hyper-Gradient Descent

Kaan Ozkara; Can Karakus; Parameswaran Raman; Mingyi Hong; Shoham Sabach; Branislav Kveton; Volkan Cevher

2024 ICML ICML 2024

MADA: Meta-Adaptive Optimizers Through Hyper-Gradient Descent

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Kaan Ozkara, Can Karakus, Parameswaran Raman, Mingyi Hong, Shoham Sabach, Branislav Kveton, Volkan Cevher

Topics

Machine Learning > Optimization & Theory > Optimization Deep Learning > Architectures > Neural Networks Artificial Intelligence > Core AI > Large Language Models

Keywords

hypergradient descent gpt2 amsgrad avgrad mada meta-adaptive optimizers

Download PDF