distributed training

Moniqua: Modulo Quantized Communication in Decentralized SGD ICML 2020 On the Acceleration of Deep Learning Model Parallelism With Staleness CVPR 2020 On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning AAAI 2020 Practical Low-Rank Communication Compression in Decentralized Deep Learning NeurIPS 2020 Quantized Compressive Sampling of Stochastic Gradients for Efficient Communication in Distributed Deep Learning AAAI 2020 ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training NeurIPS 2020 Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts NeurIPS 2020 Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training EMNLP 2019 Communication trade-offs for Local-SGD with large step size NeurIPS 2019 Deep Leakage from Gradients NeurIPS 2019 DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation NeurIPS 2019 FABA: An Algorithm for Fast Aggregation against Byzantine Attacks in Distributed Neural Networks IJCAI 2019 fairseq: A Fast, Extensible Toolkit for Sequence Modeling NAACL 2019 Faster Distributed Deep Net Training: Computation and Communication Decoupled Stochastic Gradient Descent IJCAI 2019 Fault Tolerance in Iterative-Convergent Machine Learning ICML 2019 FreeFlow: Software-based Virtual RDMA Networking for Containerized Clouds NSDI 2019 Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning NeurIPS 2019 GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism NeurIPS 2019 Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks CVPR 2019 Large-Scale Mixed-Bandwidth Deep Neural Network Acoustic Modeling for Automatic Speech Recognition Interspeech 2019 Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models NeurIPS 2019 Learning Generalizable Device Placement Algorithms for Distributed Machine Learning NeurIPS 2019 Local SGD Converges Fast and Communicates Little ICLR 2019 Optimization of Inf-Convolution Regularized Nonconvex Composite Problems AISTATS 2019 Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning AAAI 2019