Skip to content
conftrace_
2016 COLT COLT 2016

Gradient Descent Only Converges to Minimizers

Abstract

We show that gradient descent converges to a local minimizer, almost surely with random initial- ization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

🐣 Hot Topic Early Bird - dynamical system

Authors

Jason D. Lee, Max Simchowitz, Michael I. Jordan, Benjamin Recht

Topics

Machine Learning > Optimization & Theory > Optimization Machine Learning > Optimization & Theory > Theory

Keywords

gradient descent dynamical system random initialization local minimizer stable manifold
Download PDF

More from COLT 2016

All COLT 2016 papers →
Online Isotonic Regression
Time series prediction and online learning
Regret Analysis of the Finite-Horizon Gittins Index Strategy for Multi-Armed Bandits
Learning Communities in the Presence of Errors
On the capacity of information processing systems
Learning Simple Auctions