adaptive optimizer

HVAdam: A Full-Dimension Adaptive Optimizer AAAI 2025 Memory-Efficient Optimization with Factorized Hamiltonian Descent AISTATS 2025 Taming LLMs with Gradient Grouping ACL 2025 Understanding Optimization in Deep Learning with Central Flows ICLR 2025 MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence NeurIPS 2024 Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks JMLR 2024 Should I try multiple optimizers when fine-tuning a pre-trained Transformer for NLP tasks? Should I tune their hyperparameters? EACL 2024 When Will Gradient Regularization Be Harmful? ICML 2024 Adaptive Optimization in the ∞-Width Limit ICLR 2023 AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix NeurIPS 2023 Birder: Communication-Efficient 1-bit Adaptive Optimizer for Practical Distributed DNN Training NeurIPS 2023 Meta-AdaM: An Meta-Learned Adaptive Optimizer with Momentum for Few-Shot Learning NeurIPS 2023 The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection AAAI 2022 Asynchronous Stochastic Gradient Descent for Extreme-Scale Recommender Systems AAAI 2021 Momentum Centering and Asynchronous Update for Adaptive Gradient Methods NeurIPS 2021 Understanding Decoupled and Early Weight Decay AAAI 2021 L4: Practical loss-based stepsize adaptation for deep learning NeurIPS 2018 Neural Taylor Approximations: Convergence and Exploration in Rectifier Networks ICML 2017