conftrace_

batch size

34 papers

Explore in graph

Co-occurring keywords

stochastic gradient descent (1091) learning rate (192) large language model (13587) neural network (6616) gradient variance (31) neural network optimization (1293) convergence analysis (395) neural network training (309) memory optimization (114) language model (4599)

Papers

On the Generalization Benefit of Noise in Stochastic Gradient Descent ICML 2020
Measuring the Effects of Data Parallelism on Neural Network Training JMLR 2019
On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization ICML 2019
A Highly Efficient Distributed Deep Learning System for Automatic Speech Recognition INTERSPEECH 2019
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study ICML 2019
Control Batch Size and Learning Rate to Generalize Well: Theoretical and Empirical Evidence NIPS 2019
How SGD Selects the Global Minima in Over-parameterized Learning: A Dynamical Stability Perspective NIPS 2018
Efficient nonmyopic batch active search NIPS 2018
Adaptive Batch Size for Safe Policy Gradients NIPS 2017