Distributed Learning

AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving OSDI 2023 Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression: Fast Convergence and Partial Participation ICML 2023 Anchor Sampling for Federated Learning with Partial Client Participation ICML 2023 A one-sample decentralized proximal algorithm for non-convex stochastic composite optimization UAI 2023 A principled framework for the design and analysis of token algorithms AISTATS 2023 ARK: GPU-driven Code Execution for Distributed Deep Learning NSDI 2023 ASPEN: Breaking Operator Barriers for Efficient Parallelization of Deep Neural Networks NeurIPS 2023 Attacks against Federated Learning Defense Systems and their Mitigation JMLR 2023 Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning ICML 2023 Backpropagation through Combinatorial Algorithms: Identity with Projection Works ICLR 2023 Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs NSDI 2023 Better Together: Jointly Optimizing ML Collective Scheduling and Execution Planning using SYNDICATE NSDI 2023 Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence NeurIPS 2023 Beyond Spectral Gap: The Role of the Topology in Decentralized Learning JMLR 2023 Birder: Communication-Efficient 1-bit Adaptive Optimizer for Practical Distributed DNN Training NeurIPS 2023 BPipe: Memory-Balanced Pipeline Parallelism for Training Large Language Models ICML 2023 Breaking the Communication-Privacy-Accuracy Tradeoff with f-Differential Privacy NeurIPS 2023 Buffered Asynchronous SGD for Byzantine Learning JMLR 2023 Byzantine-Tolerant Methods for Distributed Variational Inequalities NeurIPS 2023 Can 5th Generation Local Training Methods Support Client Sampling? Yes! AISTATS 2023 Catalyst Acceleration of Error Compensated Methods Leads to Better Communication Complexity AISTATS 2023 Client-Customized Adaptation for Parameter-Efficient Federated Learning ACL 2023 CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks ICML 2023 ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning ACL 2023 Collaborative Learning via Prediction Consensus NeurIPS 2023