conftrace_

distributed training

188 papers

Explore in graph

Co-occurring keywords

distributed learning (563) communication efficiency (297) model parallelism (43) stochastic gradient descent (1091) gradient compression (78) neural network optimization (1293) federated learning (1333) large language model (13587) model compression (3302) neural network (6616)

Papers

SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient ICML 2023
Toward Large Kernel Models ICML 2023
Quantized Distributed Training of Large Models with Convergence Guarantees ICML 2023
Lowering the Pre-training Tax for Gradient-based Subset Training: A Lightweight Distributed Pre-Training Toolkit ICML 2023
TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches NSDI 2023
3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation ICML 2022
Survey on Efficient Training of Large Neural Networks IJCAI 2022
DRAGONN: Distributed Randomized Approximate Gradients of Neural Networks ICML 2022
Secure Distributed Training at Scale ICML 2022
WarpDrive: Fast End-to-End Deep Multi-Agent Reinforcement Learning on a GPU JMLR 2022
DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems AAAI 2022
A Deep Learning Dataloader with Shared Data Preparation NIPS 2022
A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks NIPS 2022
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness NIPS 2022
Unity: Accelerating DNN Training Through Joint Optimization of Algebraic Transformations and Parallelization OSDI 2022
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning OSDI 2022
Shifted compression framework: generalizations and improvements UAI 2022
ResIST: Layer-wise decomposition of ResNets for distributed training UAI 2022
solo-learn: A Library of Self-supervised Methods for Visual Representation Learning JMLR 2022
Interlocking Backpropagation: Improving depthwise model-parallelism JMLR 2022
FedNST: Federated Noisy Student Training for Automatic Speech Recognition INTERSPEECH 2022
Intrinsic Gradient Compression for Scalable and Efficient Federated Learning ACL 2022
Federated Learning With Position-Aware Neurons CVPR 2022
Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning NIPS 2022
GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models NIPS 2022