conftrace_

distributed training

188 papers

Explore in graph

Co-occurring keywords

distributed learning (563) communication efficiency (297) model parallelism (43) stochastic gradient descent (1091) gradient compression (78) neural network optimization (1293) federated learning (1333) large language model (13587) model compression (3302) neural network (6616)

Papers

Stochastic Sign Descent Methods: New Algorithms and Better Theory ICML 2021
Cross-Gradient Aggregation for Decentralized Learning from Non-IID Data ICML 2021
Revealing and Protecting Labels in Distributed Training NIPS 2021
WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition CVPR 2021
Communication Efficient SGD via Gradient Sampling With Bayes Prior CVPR 2021
Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads OSDI 2021
Memory-Efficient Pipeline-Parallel DNN Training ICML 2021
On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning AAAI 2020
Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts NIPS 2020
KungFu: Making Training in Distributed Machine Learning Adaptive OSDI 2020
Indirect Stochastic Gradient Quantization and Its Application in Distributed Deep Learning AAAI 2020
Moniqua: Modulo Quantized Communication in Decentralized SGD ICML 2020
On the Acceleration of Deep Learning Model Parallelism With Staleness CVPR 2020
A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters OSDI 2020
CDC: Classification Driven Compression for Bandwidth Efficient Edge-Cloud Collaborative Deep Learning IJCAI 2020
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training NIPS 2020
Practical Low-Rank Communication Compression in Decentralized Deep Learning NIPS 2020
Decentralized gradient methods: does topology matter? AISTATS 2020
AdaScale SGD: A User-Friendly Algorithm for Distributed Training ICML 2020
Quantized Compressive Sampling of Stochastic Gradients for Efficient Communication in Distributed Deep Learning AAAI 2020
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms NIPS 2020
Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning NIPS 2019
Large-Scale Mixed-Bandwidth Deep Neural Network Acoustic Modeling for Automatic Speech Recognition INTERSPEECH 2019
Two Tiered Distributed Training Algorithm for Acoustic Modeling INTERSPEECH 2019
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training IJCNLP 2019