conftrace_

adam optimizer

54 papers

Explore in graph

Also known as

ADAM

Co-occurring keywords

stochastic gradient descent (1091) neural network optimization (1293) adaptive optimization (52) gradient descent (1144) stochastic optimization (1060) learning rate (192) convergence guarantee (169) adaptive learning rate (58) convergence rate (607) hyperparameter tuning (215)

Papers

How Does Adaptive Optimization Impact Local Neural Network Geometry? NIPS 2023
Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models NIPS 2023
The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection AAAI 2022
Adam Can Converge Without Any Modification On Update Rules NIPS 2022
A Momentumized, Adaptive, Dual Averaged Gradient Method JMLR 2022
Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration JMLR 2022
VectorAdam for Rotation Equivariant Geometry Optimization NIPS 2022
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity AAAI 2022
Adaptive Differential Privacy for Language Model Training ACL 2022
On the distributional properties of adaptive gradients UAI 2021
How Do Adam and Training Strategies Help BNNs Optimization ICML 2021
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers ICML 2021
The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks ICML 2021
On the Adequacy of Untuned Warmup for Adaptive Optimization AAAI 2021
1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed ICML 2021
MTAdam: Automatic Balancing of Multiple Training Loss Terms EMNLP 2021
Domain-Independent Dominance of Adaptive Methods CVPR 2021
GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training NIPS 2021
Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods NIPS 2020
Analysis of Q-learning with Adaptation and Momentum Restart for Gradient Descent IJCAI 2020
Towards Better Generalization of Adaptive Gradient Methods NIPS 2020
Towards Theoretically Understanding Why Sgd Generalizes Better Than Adam in Deep Learning NIPS 2020
Adam with Bandit Sampling for Deep Learning NIPS 2020
HyperAdam: A Learnable Task-Adaptive Adam for Network Training AAAI 2019
Memory Efficient Adaptive Optimization NIPS 2019