conftrace_

momentum parameter

3 papers

Explore in graph

Co-occurring keywords · papers mentioning both

stochastic gradient descent (2) heavy ball momentum (1) quadratic convex objective (1) descending warmup (1) averaged solution (1) adaptive polyak's heavy-ball (1) individual convergence (1) exponential moving average (1) critical point (1) adam optimizer (1)

Papers

Exploring the Inefficiency of Heavy Ball as Momentum Parameter Approaches 1 IJCAI 2024 Adam Can Converge Without Any Modification On Update Rules NeurIPS 2022 The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods ICLR 2021