2021
COLT
COLT 2021
The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication
Abstract
We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where $M$ machines work in parallel over the course of $R$ rounds of communication to optimize the objective, and during each round of communication, each machine may sequentially compute $K$ stochastic gradient estimates. We present a novel lower bound with a matching upper bound that establishes an optimal algorithm.
๐
Interdisciplinary Bridge
- Machine Learning and Mathematics & Optimization
๐งญ
Keyword Pioneer
- intermittent communication
๐ฃ
Hot Topic Early Bird
- lower bound
๐
Cross-Pollinator
- Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio