Skip to content
conftrace_

gpt training

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

learning rate warmup (1) critical batch size (1) adamw optimizer (1) adam optimizer (1) gradient descent (1) neural network optimization (1) representation learning (1)

Papers

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training NeurIPS 2024