Distributed Learning

Sketching for Distributed Deep Learning: A Sharper Analysis NeurIPS 2024 SLowcalSGD : Slow Query Points Improve Local-SGD for Stochastic Convex Optimization NeurIPS 2024 SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization NeurIPS 2024 SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation NeurIPS 2024 Stabilized Proximal-Point Methods for Federated Optimization NeurIPS 2024 Stochastic Controlled Averaging for Federated Learning with Communication Compression ICLR 2024 THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression NSDI 2024 The Bedrock of Byzantine Fault Tolerance: A Unified Platform for BFT Protocols Analysis, Implementation, and Experimentation NSDI 2024 The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication COLT 2024 The Promise of Serverless Computing within Peer-to-Peer Architectures for Distributed ML Training AAAI 2024 Thinking Forward: Memory-Efficient Federated Finetuning of Language Models NeurIPS 2024 Tolerating Outliers: Gradient-Based Penalties for Byzantine Robustness and Inclusion IJCAI 2024 Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent AAAI 2024 Traceable Federated Continual Learning CVPR 2024 Universal Exact Compression of Differentially Private Mechanisms NeurIPS 2024 Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning CVPR 2024 Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML NeurIPS 2024 What's In My Big Data? ICLR 2024 Zero Bubble (Almost) Pipeline Parallelism ICLR 2024 ZeRO++: Extremely Efficient Collective Communication for Large Model Training ICLR 2024 A^2CiD^2: Accelerating Asynchronous Communication in Decentralized Deep Learning NeurIPS 2023 2Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression NeurIPS 2023 Accelerated Primal-Dual Mirror Dynamics for Centralized and Distributed Constrained Convex Optimization Problems JMLR 2023 A Computation and Communication Efficient Method for Distributed Nonconvex Problems in the Partial Participation Setting NeurIPS 2023 Aggregating Capacity in FL through Successive Layer Training for Computationally-Constrained Devices NeurIPS 2023