conftrace_

adam optimizer

54 papers

Explore in graph

Also known as

ADAM

Co-occurring keywords

stochastic gradient descent (1091) neural network optimization (1293) adaptive optimization (52) gradient descent (1144) stochastic optimization (1060) learning rate (192) convergence guarantee (169) adaptive learning rate (58) convergence rate (607) hyperparameter tuning (215)

Papers

How to Set $\beta_1, \beta_2$ in Adam: An Online Learning Perspective ALT 2026
AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training EMNLP 2025
Preconditioners for the Stochastic Training of Neural Fields CVPR 2025
A Multi-Granularity Clustering Approach for Federated Backdoor Defense with the Adam Optimizer IJCAI 2025
Better Embeddings with Coupled Adam ACL 2025
BSFA: Leveraging the Subspace Dichotomy to Accelerate Neural Network Training EMNLP 2025
ADOPT: Modified Adam Can Converge with Any $\beta_2$ with the Optimal Rate NIPS 2024
Why Warmup the Learning Rate? Underlying Mechanisms and Improvements NIPS 2024
Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling NIPS 2024
Why Transformers Need Adam: A Hessian Perspective NIPS 2024
BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models NIPS 2024
Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training NIPS 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions NIPS 2024
Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees JMLR 2024
DP-AdamBC: Your DP-Adam Is Actually DP-SGD (Unless You Apply Bias Correction) AAAI 2024
The Implicit Bias of Adam on Separable Data NIPS 2024
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence NIPS 2024
Boosting Diffusion Models with an Adaptive Momentum Sampler IJCAI 2024
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models NIPS 2024
Adam with model exponential moving average is effective for nonconvex optimization NIPS 2024
How Well Can a Genetic Algorithm Fine-tune Transformer Encoders? A First Approach NAACL 2024
How Does Adaptive Optimization Impact Local Neural Network Geometry? NIPS 2023
Closing the gap between the upper bound and lower bound of Adam's iteration complexity NIPS 2023
AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix NIPS 2023
STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition ICML 2023