distributed training

Communication Efficient Distributed Training with Distributed Lion NeurIPS 2024 Communication-Efficient Federated Learning With Data and Client Heterogeneity AISTATS 2024 DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models CVPR 2024 DISTMM: Accelerating Distributed Multimodal Model Training NSDI 2024 DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus NeurIPS 2024 Don't Compress Gradients in Random Reshuffling: Compress Gradient Differences NeurIPS 2024 EControl: Fast Distributed Optimization with Compression and Error Control ICLR 2024 Error Feedback Reloaded: From Quadratic to Arithmetic Mean of Smoothness Constants ICLR 2024 FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-Aware Model Update AAAI 2024 Flag Aggregator: Scalable Distributed Training under Failures and Augmented Losses using Convex Optimization ICLR 2024 GRAWA: Gradient-based Weighted Averaging for Distributed Training of Deep Learning Models AISTATS 2024 Incorporating Serverless Computing into P2P Networks for ML Training: In-Database Tasks and Their Scalability Implications (Student Abstract) AAAI 2024 JORA: JAX Tensor-Parallel LoRA Library for Retrieval Augmented Fine-Tuning ACL 2024 LocMoE: A Low-overhead MoE for Large Language Model Training IJCAI 2024 LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing NeurIPS 2024 MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale OSDI 2024 MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts NeurIPS 2024 Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances NSDI 2024 Pipeline Parallelism with Controllable Memory NeurIPS 2024 RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs NAACL 2024 Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models NeurIPS 2024 SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training NeurIPS 2024 SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation NeurIPS 2024 THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression NSDI 2024 The Promise of Serverless Computing within Peer-to-Peer Architectures for Distributed ML Training AAAI 2024