conftrace_

2009 NeurIPS NeurIPS 2009

Slow Learners are Fast

Abstract

Online learning algorithms have impressive convergence properties when it comes to risk minimization and convex games on very large problems. However, they are inherently sequential in their design which prevents them from taking advantage of modern multi-core architectures. In this paper we prove that online learning with delayed updates converges well, thereby facilitating parallel online learning.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Mathematics & Optimization

📈 Trend Setter - Convex Optimization

🐣 Hot Topic Early Bird - convex optimization

Authors

Martin Zinkevich, John Langford, Alex J. Smola

Topics

Machine Learning > Optimization & Theory > Distributed Learning Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Optimization & Theory > Optimization Mathematics & Optimization > Optimization > Online Algorithms Machine Learning > Learning Types > Online Learning Machine Learning > Optimization & Theory > Online Algorithms Machine Learning > Optimization & Theory > Stochastic Methods Deep Learning > Optimization & Theory > Efficient Computing Machine Learning > Optimization & Theory > Convex Optimization

Keywords

stochastic optimization online learning convex optimization risk minimization parallel learning delayed update parallel online learning convex game multi-core architecture

More from NeurIPS 2009

All NeurIPS 2009 papers →

Sufficient Conditions for Agnostic Active Learnable

The Ordered Residual Kernel for Robust Motion Subspace Clustering

Learning to Rank by Optimizing NDCG Measure

Tracking Dynamic Sources of Malicious Activity at Internet Scale

Graph Zeta Function in the Bethe Free Energy and Loopy Belief Propagation

Learning in Markov Random Fields using Tempered Transitions