conftrace_

distributed training

188 papers

Explore in graph

Co-occurring keywords

distributed learning (563) communication efficiency (297) model parallelism (43) stochastic gradient descent (1091) gradient compression (78) neural network optimization (1293) federated learning (1333) large language model (13587) model compression (3302) neural network (6616)

Papers

FLAIR: Federated Learning Annotated Image Repository NIPS 2022
VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangement NIPS 2022
Quantized Training of Gradient Boosting Decision Trees NIPS 2022
SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training NIPS 2022
TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training NIPS 2022
Unlocking the Power of Inline Floating-Point Operations on Programmable Switches NSDI 2022
On the Convergence of Decentralized Adaptive Gradient Methods ACML 2022
Gradient Forward-Propagation for Large-Scale Temporal Video Modelling CVPR 2021
1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed ICML 2021
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization JMLR 2021
HetSeq: Distributed GPU Training on Heterogeneous Infrastructure AAAI 2021
Error Compensated Distributed SGD Can Be Accelerated NIPS 2021
Rethinking gradient sparsification as total error minimization NIPS 2021
Training Neural Networks with Fixed Sparse Masks NIPS 2021
EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback NIPS 2021
PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models ICML 2021
Accelerating Gossip SGD with Periodic Global Averaging ICML 2021
NeurST: Neural Speech Translation Toolkit IJCNLP 2021
NeurST: Neural Speech Translation Toolkit ACL 2021
P3: Distributed Deep Graph Learning at Scale OSDI 2021
Oort: Efficient Federated Learning via Guided Participant Selection OSDI 2021
Constrained Robust Submodular Partitioning NIPS 2021
TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models ICML 2021
Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees AAAI 2021
Step-Ahead Error Feedback for Distributed Training with Compressed Gradient AAAI 2021