Skip to content
conftrace_
2016 ICML ICML 2016

Solving Ridge Regression using Sketched Preconditioned SVRG

Abstract

We develop a novel preconditioning method for ridge regression, based on recent linear sketching methods. By equipping Stochastic Variance Reduced Gradient (SVRG) with this preconditioning process, we obtain a significant speed-up relative to fast stochastic methods such as SVRG, SDCA and SAG.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Mathematics & Optimization

Authors

Alon Gonen, Francesco Orabona, Shai Shalev-shwartz

Topics

Machine Learning > Core Methods > Regression Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Optimization & Theory > Optimization Mathematics & Optimization > Mathematics > Numerical Analysis Mathematics & Optimization > Optimization > Continuous Optimization

Keywords

stochastic gradient descent ridge regression numerical optimization stochastic variance reduced gradient linear sketching
Download PDF

More from ICML 2016

All ICML 2016 papers →
On the Statistical Limits of Convex Relaxations
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
Gossip Dual Averaging for Decentralized Optimization of Pairwise Functions
Cumulative Prospect Theory Meets Reinforcement Learning: Prediction and Control
Estimating Accuracy from Unlabeled Data: A Bayesian Approach
Non-negative Matrix Factorization under Heavy Noise