Skip to content
conftrace_

pre-training stability

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

embedding variability (1) multi-head low-rank attention (1) model perplexity (1) token embedding (1) language model pretraining (1) training stability (1) neural network optimization (1) attention mechanism (1)

Papers

Stable Language Model Pre-training by Reducing Embedding Variability EMNLP 2024