Skip to content
conftrace_

chinchilla-optimal language model

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

batch size scheduling (1) transformer model (1) batch size (1) batch size schedule (1) gradient noise scale (1) per-example gradient (1) layer normalization (1) transformer training (1) neural network optimization (1)

Papers

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NeurIPS 2024