Skip to content
conftrace_

data-parallel training

9 papers

Explore in graph

Co-occurring keywords · papers mentioning both

communication efficiency (4) distributed training (4) gradient quantization (2) collective communication (2) distributed learning (2) stochastic gradient descent (2) local sgd (1) large language model (1) synchronization strategy (1) layer-wise decomposition (1)

Papers

Checkmate: Zero Performance Overhead Model Checkpointing via Network Gradient Replication NSDI 2026 CO2: Efficient Distributed Training with Full Communication-Computation Overlap ICLR 2024 Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models NeurIPS 2024 Accelerating Collective Communication in Data Parallel Training across Deep Learning Frameworks NSDI 2022 ResIST: Layer-wise decomposition of ResNets for distributed training UAI 2022 NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization JMLR 2021 Adaptive Gradient Quantization for Data-Parallel SGD NeurIPS 2020 AutoSync: Learning to Synchronize for Data-Parallel Distributed Deep Learning NeurIPS 2020 PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization NeurIPS 2019