conftrace_

heavy-tailed gradient noise

2 papers

Explore in graph

Co-occurring keywords · papers mentioning both

stochastic gradient descent (2) underdamped langevin dynamics (1) flat minima (1) generalization gap (1) stochastic differential equation (1) heavy-tailed gradient (1) gradient clipping (1) adam optimizer (1) gibbs distribution (1) natural gradient (1)

Papers

Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise ICML 2020 Towards Theoretically Understanding Why Sgd Generalizes Better Than Adam in Deep Learning NeurIPS 2020