conftrace_

gradient norm

39 papers

Explore in graph

Also known as

GRADNORM GNS

Co-occurring keywords

neural network optimization (1293) stochastic gradient descent (1091) convex optimization (1321) non-convex optimization (547) model generalization (190) flat minima (50) neural network (6616) convergence rate (607) gradient descent (1144) sharpness-aware minimization (81)

Papers

Pruning has a disparate impact on model accuracy NIPS 2022
Revisit last-iterate convergence of mSGD under milder requirement on step size NIPS 2022
Accelerated Algorithms for Smooth Convex-Concave Minimax Problems with O(1/k^2) Rate on Squared Gradient Norm ICML 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training NIPS 2021
A Geometric Structure of Acceleration and Its Role in Making Gradients Small Fast NIPS 2021
On the Importance of Gradients for Detecting Distributional Shifts in the Wild NIPS 2021
Fast and Memory Efficient Differentially Private-SGD via JL Projections NIPS 2021
Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning NIPS 2021
Parameter-free, Dynamic, and Strongly-Adaptive Online Learning ICML 2020
The Complexity of Finding Stationary Points with Stochastic Gradient Descent ICML 2020
Sorting Out Lipschitz Function Approximation ICML 2019
DINGO: Distributed Newton-Type Method for Gradient-Norm Optimization NIPS 2019
How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD NIPS 2018
BING: Binarized Normed Gradients for Objectness Estimation at 300fps CVPR 2014