conftrace_

adamw

6 papers

Explore in graph

Co-occurring keywords · papers mentioning both

gpt2 (2) sgd (2) lion (2) weight decay (2) preconditioners (1) accelerated √l_p steepest descent (1) interpolated primal-dual iterate sequences (1) non-euclidean smooth optimization (1) ell_infty norm (1) frank-wolfe (1)

Papers

How to set AdamW’s weight decay as you scale model and dataset size ICML 2025 MARS: Unleashing the Power of Variance Reduction for Training Large Models ICML 2025 Stacey: Promoting Stochastic Steepest Descent via Accelerated ℓ_p-Smooth Nonconvex Optimization ICML 2025 The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training ICML 2025 Implicit Bias of AdamW: ℓ_∞-Norm Constrained Optimization ICML 2024 Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models ICML 2024