batch size

Existence and Estimation of Critical Batch Size for Training Generative Adversarial Networks with Two Time-Scale Update Rule ICML 2023 Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions ICML 2023 Sigmoid Loss for Language Image Pre-Training ICCV 2023 Simplifying Neural Network Training Under Class Imbalance NeurIPS 2023 Small batch deep reinforcement learning NeurIPS 2023 Batch size-invariance for policy optimization NeurIPS 2022 Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training JMLR 2022 The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models NeurIPS 2022 Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions NeurIPS 2022 A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima ICLR 2021 Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates AISTATS 2021 Remember More by Recalling Less: Investigating the Role of Batch Size in Continual Learning with Experience Replay (Student Abstract) AAAI 2021 Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup ACL 2021 SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement NeurIPS 2021 Understanding Gradient Clipping In Incremental Gradient Methods AISTATS 2021 An Empirical Study of Stochastic Gradient Descent with Structured Covariance Noise AISTATS 2020 On the Generalization Benefit of Noise in Stochastic Gradient Descent ICML 2020 Stochastic Optimization with Laggard Data Pipelines NeurIPS 2020 Understanding the Effects of Batching in Online Active Learning AISTATS 2020 A Highly Efficient Distributed Deep Learning System for Automatic Speech Recognition Interspeech 2019 Control Batch Size and Learning Rate to Generalize Well: Theoretical and Empirical Evidence NeurIPS 2019 Measuring the Effects of Data Parallelism on Neural Network Training JMLR 2019 On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization ICML 2019 The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study ICML 2019 A Bayesian Perspective on Generalization and Stochastic Gradient Descent ICLR 2018