conftrace_

gradient descent

1144 papers

Explore in graph

Also known as

GD

Co-occurring keywords

neural network (6616) neural network optimization (1293) convex optimization (1321) non-convex optimization (547) convergence rate (607) stochastic optimization (1060) convergence analysis (395) stochastic gradient descent (1091) nonconvex optimization (316) implicit bia (107)

Papers

A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions JMLR 2022
Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration JMLR 2022
I AM A.I. Gradient Descent – an Open-Source Digital Game for Inquiry-Based CLIL Learning AAAI 2022
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms NIPS 2022
Symmetry Teleportation for Accelerated Optimization NIPS 2022
On the non-universality of deep learning: quantifying the cost of symmetry NIPS 2022
Large-scale Optimization of Partial AUC in a Range of False Positive Rates NIPS 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation NIPS 2022
Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis NIPS 2022
Do Residual Neural Networks discretize Neural Ordinary Differential Equations? NIPS 2022
A Parametric Class of Approximate Gradient Updates for Policy Optimization ICML 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent NIPS 2022
Multi-Objective Deep Learning with Adaptive Reference Vectors NIPS 2022
Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently NIPS 2022
Benign Overfitting in Two-layer Convolutional Neural Networks NIPS 2022
Globally Convergent Policy Search for Output Estimation NIPS 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning NIPS 2022
Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules NIPS 2022
Surprising Instabilities in Training Deep Networks and a Theoretical Analysis NIPS 2022
Towards Theoretically Inspired Neural Initialization Optimization NIPS 2022
Biologically-plausible backpropagation through arbitrary timespans via local neuromodulators NIPS 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials NIPS 2022
Learning Distributions Generated by Single-Layer ReLU Networks in the Presence of Arbitrary Outliers NIPS 2022
Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability NIPS 2022
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization NIPS 2022