Skip to content
conftrace_

batch size scaling

5 papers

Explore in graph

Co-occurring keywords · papers mentioning both

scaling rule (1) optimizer theory (1) layerwise adaptive learning rate (1) learning rate scaling (1) clip training (1) memory efficient (1) bert pretraining (1) large batch optimization (1) large-scale training (1) exponential moving average (1)

Papers

Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling NeurIPS 2024 DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training CVPR 2023 How to Scale Your EMA NeurIPS 2023 On the SDEs and Scaling Rules for Adaptive Gradient Algorithms NeurIPS 2022 Large Batch Optimization for Deep Learning: Training BERT in 76 minutes ICLR 2020