model parallelism

DeToNATION: Decoupled Torch Network-Aware Training on Interlinked Online Nodes AAAI 2026 FusedRec: Fused Embedding Communication for Distributed Recommendation Training on GPUs AAAI 2026 Revisiting Pipeline Parallelism for LLM Serving OSDI 2026 TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training AAAI 2026 Beyond A Single AI Cluster: A Survey of Decentralized LLM Training EMNLP 2025 ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development NSDI 2025 Energy Considerations of Large Language Model Inference and Efficiency Optimizations ACL 2025 FoldMoE: Efficient Long Sequence MoE Training via Attention-MoE Pipelining ACL 2025 Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping ICML 2025 MoEP: Modular Expert Paths for Sample-Efficient Language Modeling EMNLP 2025 PETRA: Parallel End-to-end Training with Reversible Architectures ICLR 2025 Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters AAAI 2025 SimAI: Unifying Architecture Design and Performance Tuning for Large-Scale Large Language Model Training with Scalability and Precision NSDI 2025 AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising NeurIPS 2024 CO2: Efficient Distributed Training with Full Communication-Computation Overlap ICLR 2024 DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models CVPR 2024 DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving OSDI 2024 LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing NeurIPS 2024 Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances NSDI 2024 Pipeline Parallelism with Controllable Memory NeurIPS 2024 RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs NAACL 2024 Towards a Better Theoretical Understanding of Independent Subnetwork Training ICML 2024 AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving OSDI 2023 H3T: Efficient Integration of Memory Optimization and Parallelism for Large-scale Transformer Training NeurIPS 2023 OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning IJCAI 2023