conftrace_

distributed training

188 papers

Explore in graph

Co-occurring keywords

distributed learning (563) communication efficiency (297) model parallelism (43) stochastic gradient descent (1091) gradient compression (78) neural network optimization (1293) federated learning (1333) large language model (13587) model compression (3302) neural network (6616)

Papers

ATOMO: Communication-efficient Learning via Atomic Sparsification NIPS 2018
The Convergence of Sparsified Gradient Methods NIPS 2018
OpenSeq2Seq: Extensible Toolkit for Distributed and Mixed Precision Training of Sequence-to-Sequence Models ACL 2018
Mesh-TensorFlow: Deep Learning for Supercomputers NIPS 2018
DRACO: Byzantine-resilient Distributed Training via Redundant Gradients ICML 2018
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning NIPS 2017
FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters CVPR 2016
TensorFlow: A System for Large-Scale Machine Learning OSDI 2016
Large-scale Distributed Dependent Nonparametric Trees ICML 2015
Deep learning with COTS HPC systems ICML 2013
Distributed training of Large-scale Logistic models ICML 2013
Large Scale Distributed Deep Networks NIPS 2012
Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models NIPS 2009