Skip to content
conftrace_

multi-head low-rank attention

1 paper

Explore in graph

Also known as

MHLRA

Co-occurring keywords · papers mentioning both

pre-training stability (1) embedding variability (1) model perplexity (1) token embedding (1) language model pretraining (1) training stability (1) neural network optimization (1) attention mechanism (1)

Papers

Stable Language Model Pre-training by Reducing Embedding Variability EMNLP 2024