Skip to content
conftrace_

gradient noise scale

1 paper

Explore in graph

Also known as

GNS

Co-occurring keywords · papers mentioning both

per-example gradient (1) layer normalization (1) batch size schedule (1) batch size (1) transformer model (1) batch size scheduling (1) chinchilla-optimal language model (1) transformer training (1) neural network optimization (1)

Papers

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NeurIPS 2024