Skip to content
conftrace_

batch size scheduling

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

chinchilla-optimal language model (1) transformer model (1) batch size (1) batch size schedule (1) gradient noise scale (1) per-example gradient (1) layer normalization (1) transformer training (1) neural network optimization (1)

Papers

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NeurIPS 2024