conftrace_

parameter norm growth

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

saturated network (1) parameter norm (1) discrete structure (1) self attention (1) transformer training (1) language model (1) inductive bia (1) gradient descent (1) attention mechanism (1)

Papers

Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent EMNLP 2021