Skip to content
conftrace_

per-example gradient

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

gradient noise scale (1) layer normalization (1) batch size schedule (1) batch size (1) transformer model (1) batch size scheduling (1) chinchilla-optimal language model (1) transformer training (1) neural network optimization (1)

Papers

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NeurIPS 2024