Skip to content
conftrace_

learning rate scaling

6 papers

Explore in graph

Co-occurring keywords · papers mentioning both

large batch training (2) adaptive learning rate (2) layer-wise adaptive rate scaling (1) layer-wise scaling (1) batch size scaling (1) data up-sampling (1) optimizer theory (1) adamw (1) parameterization (1) adam-atan2 (1)

Papers

How to set AdamW’s weight decay as you scale model and dataset size ICML 2025 Scaling Exponents Across Parameterizations and Optimizers ICML 2024 Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling NeurIPS 2024 Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages EACL 2023 Large Batch Optimization for Deep Learning Using New Complete Layer-Wise Adaptive Rate Scaling AAAI 2021 AdaScale SGD: A User-Friendly Algorithm for Distributed Training ICML 2020