distributed training

Gradient Forward-Propagation for Large-Scale Temporal Video Modelling CVPR 2021 HetSeq: Distributed GPU Training on Heterogeneous Infrastructure AAAI 2021 Memory-Efficient Pipeline-Parallel DNN Training ICML 2021 NeurST: Neural Speech Translation Toolkit ACL 2021 NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization JMLR 2021 Oort: Efficient Federated Learning via Guided Participant Selection OSDI 2021 P3: Distributed Deep Graph Learning at Scale OSDI 2021 PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models ICML 2021 Rethinking gradient sparsification as total error minimization NeurIPS 2021 Revealing and Protecting Labels in Distributed Training NeurIPS 2021 SOLAR: Sparse Orthogonal Learned and Random Embeddings ICLR 2021 Step-Ahead Error Feedback for Distributed Training with Compressed Gradient AAAI 2021 Stochastic Sign Descent Methods: New Algorithms and Better Theory ICML 2021 TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models ICML 2021 Training Neural Networks with Fixed Sparse Masks NeurIPS 2021 WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition CVPR 2021 AdaScale SGD: A User-Friendly Algorithm for Distributed Training ICML 2020 A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters OSDI 2020 CDC: Classification Driven Compression for Bandwidth Efficient Edge-Cloud Collaborative Deep Learning IJCAI 2020 Decentralized gradient methods: does topology matter? AISTATS 2020 Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms NeurIPS 2020 Don't Use Large Mini-batches, Use Local SGD ICLR 2020 Indirect Stochastic Gradient Quantization and Its Application in Distributed Deep Learning AAAI 2020 KungFu: Making Training in Distributed Machine Learning Adaptive OSDI 2020 Learning Efficient Parameter Server Synchronization Policies for Distributed SGD ICLR 2020