data parallelism

Accelerating Diffusion via Hybrid Data-Pipeline Parallelism Based on Conditional Guidance Scheduling CVPR 2026 BitDP: Ultra-low-bit Communication for Data Parallelism in LLM Training AAAI 2026 How Does Critical Batch Size Scale in Pre-training? ICLR 2025 LegoMT2: Selective Asynchronous Sharded Data Parallel Training for Massive Neural Machine Translation ACL 2025 Optimal Distributed Training With Co-Adaptive Data Parallelism in Heterogeneous Environments IJCAI 2025 Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters AAAI 2025 A Quadratic Synchronization Rule for Distributed Deep Learning ICLR 2024 DISTMM: Accelerating Distributed Multimodal Model Training NSDI 2024 Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models NeurIPS 2024 H3T: Efficient Integration of Memory Optimization and Parallelism for Large-scale Transformer Training NeurIPS 2023 OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning IJCAI 2023 Decentralized Training of Foundation Models in Heterogeneous Environments NeurIPS 2022 Mega-NERF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs CVPR 2022 SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training NeurIPS 2022 Understanding the effects of data parallelism and sparsity on neural network training ICLR 2021 Measuring the Effects of Data Parallelism on Neural Network Training JMLR 2019 Ouroboros: On Accelerating Training of Transformer-Based Language Models NeurIPS 2019 Exploring Hidden Dimensions in Accelerating Convolutional Neural Networks ICML 2018 Mesh-TensorFlow: Deep Learning for Supercomputers NeurIPS 2018 On-the-fly Operation Batching in Dynamic Computation Graphs NeurIPS 2017 TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning NeurIPS 2017 On Model Parallelization and Scheduling Strategies for Distributed Machine Learning NeurIPS 2014 Large Scale Distributed Deep Networks NeurIPS 2012