adam optimizer

How to Set β_1, β_2 in Adam: An Online Learning Perspective ALT 2026 The Convergence Behavior of Adam under Heavy-Tailed Noise UAI 2026 Accelerating Training with Neuron Interaction and Nowcasting Networks ICLR 2025 Adam Exploits ℓ_∞-geometry of Loss Landscape via Coordinate-wise Adaptivity ICLR 2025 ADAM Optimization with Adaptive Batch Selection ICLR 2025 AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training EMNLP 2025 A Multi-Granularity Clustering Approach for Federated Backdoor Defense with the Adam Optimizer IJCAI 2025 Better Embeddings with Coupled Adam ACL 2025 BSFA: Leveraging the Subspace Dichotomy to Accelerate Neural Network Training EMNLP 2025 Deconstructing What Makes a Good Optimizer for Autoregressive Language Models ICLR 2025 Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator ICML 2025 Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixtures ICLR 2025 On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent ICLR 2025 Preconditioners for the Stochastic Training of Neural Fields CVPR 2025 Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees JMLR 2024 Adam with model exponential moving average is effective for nonconvex optimization NeurIPS 2024 ADOPT: Modified Adam Can Converge with Any β_2 with the Optimal Rate NeurIPS 2024 Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training NeurIPS 2024 BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models NeurIPS 2024 Boosting Diffusion Models with an Adaptive Momentum Sampler IJCAI 2024 DP-AdamBC: Your DP-Adam Is Actually DP-SGD (Unless You Apply Bias Correction) AAAI 2024 Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models NeurIPS 2024 How Well Can a Genetic Algorithm Fine-tune Transformer Encoders? A First Approach NAACL 2024 LESS: Selecting Influential Data for Targeted Instruction Tuning ICML 2024 MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence NeurIPS 2024