distributed training

Attack of the Bubbles: Straggler-Resilient Pipeline Parallelism for Large Model Training NSDI 2026 BitDP: Ultra-low-bit Communication for Data Parallelism in LLM Training AAAI 2026 Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning UAI 2026 DeToNATION: Decoupled Torch Network-Aware Training on Interlinked Online Nodes AAAI 2026 Di-PS: System-Algorithm Co-Design for Asynchronous and Heterogeneous Cross-cluster LLM Training at Scale NSDI 2026 DynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training OSDI 2026 FLARE: Anomaly Diagnostics for Divergent LLM Training in GPU Clusters of Thousand-Plus Scale NSDI 2026 FusedRec: Fused Embedding Communication for Distributed Recommendation Training on GPUs AAAI 2026 Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations OSDI 2026 OmniScale: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo AAAI 2026 OptiCo: Adaptive Distributed Training Optimization via Collaborative Agent Reasoning ACL 2026 RLBoost: Harvesting Preemptible Cloud Resources for Cost-Efficient Reinforcement Learning on LLMs NSDI 2026 Sparse Checkpointing for Fast and Reliable MoE Training NSDI 2026 Supercharging Packet-level Network Simulation of Large Model Training via Memoization and Fast-Forwarding NSDI 2026 SYMI: Efficient Mixture-of-Experts Training via Model and Optimizer State Decoupling NSDI 2026 TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training AAAI 2026 Tessera: A Holistic Pipeline Parallelism Framework for Trillion-Parameter Heterogeneous MoE Training (Operational Systems) OSDI 2026 TrainMover: An Interruption-Resilient Runtime for ML Training OSDI 2026 ActionStudio: A Lightweight Framework for Data and Training of Large Action Models EMNLP 2025 A Proactive Reliability Metric for Detecting Failures in Language Model Training EMNLP 2025 AutoCCL: Automated Collective Communication Tuning for Accelerating Distributed and Parallel DNN Training NSDI 2025 Beyond A Single AI Cluster: A Survey of Decentralized LLM Training EMNLP 2025 Breaking the Memory Barrier of Contrastive Loss via Tile-Based Strategy CVPR 2025 ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development NSDI 2025 Decentralized Diffusion Models CVPR 2025