conftrace_

effective learning rate

3 papers

Explore in graph

Also known as

ELR

Co-occurring keywords · papers mentioning both

scale-invariant neural network (1) spherical optimization (1) critical learning rate (1) elr ratios (1) trainability (1) weight projection (1) gradient explosion (1) gradient dynamics (1) weight dynamics (1) gradient descent dynamics (1)

Papers

Normalization and effective learning rates in reinforcement learning NeurIPS 2024 On the Weight Dynamics of Deep Normalized Networks ICML 2024 Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes NeurIPS 2022