Distributed Learning

FedNST: Federated Noisy Student Training for Automatic Speech Recognition Interspeech 2022 FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction NeurIPS 2022 FedScale: Benchmarking Model and System Performance of Federated Learning at Scale ICML 2022 Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees NeurIPS 2022 FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning AISTATS 2022 FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models NAACL 2022 GAL: Gradient Assisted Learning for Decentralized Multi-Organization Collaborations NeurIPS 2022 Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers ICML 2022 GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models NeurIPS 2022 Gradient and Projection Free Distributed Online Min-Max Resource Optimization L4DC 2022 Graham: Synchronizing Clocks by Leveraging Local Clock Properties NSDI 2022 Group-based Interleaved Pipeline Parallelism for Large-scale DNN Training ICLR 2022 Hierarchical Agglomerative Graph Clustering in Poly-Logarithmic Depth NeurIPS 2022 Hybrid Local SGD for Federated Learning with Heterogeneous Communications ICLR 2022 Improving Dynamic Regret in Distributed Online Mirror Descent Using Primal and Dual Information L4DC 2022 Interlocking Backpropagation: Improving depthwise model-parallelism JMLR 2022 Layer-Wised Model Aggregation for Personalized Federated Learning CVPR 2022 Learning Linear Models Using Distributed Iterative Hessian Sketching L4DC 2022 Learning To Collaborate in Decentralized Learning of Personalized Models CVPR 2022 Lightweight Projective Derivative Codes for Compressed Asynchronous Gradient Descent ICML 2022 Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression NeurIPS 2022 Massively Parallel k-Means Clustering for Perturbation Resilient Instances ICML 2022 Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averaging NeurIPS 2022 Multi-agent Performative Prediction with Greedy Deployment and Consensus Seeking Agents NeurIPS 2022 Near-Optimal Private and Scalable k-Clustering NeurIPS 2022