conftrace_

distributed training

188 papers

Explore in graph

Co-occurring keywords

distributed learning (563) communication efficiency (297) model parallelism (43) stochastic gradient descent (1091) gradient compression (78) neural network optimization (1293) federated learning (1333) large language model (13587) model compression (3302) neural network (6616)

Papers

Cheaper and Faster: Distributed Deep Reinforcement Learning with Serverless Computing AAAI 2024
Quantized Distributed Training of Large Models with Convergence Guarantees ICML 2023
Toward Large Kernel Models ICML 2023
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs NSDI 2023
trlX: A Framework for Large Scale Open Source RLHF EMNLP 2023
trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback EMNLP 2023
Global Update Tracking: A Decentralized Learning Algorithm for Heterogeneous Data NIPS 2023
H3T: Efficient Integration of Memory Optimization and Parallelism for Large-scale Transformer Training NIPS 2023
Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning NSDI 2023
Lowering the Pre-training Tax for Gradient-based Subset Training: A Lightweight Distributed Pre-Training Toolkit ICML 2023
Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-per-Second CVPR 2023
LOFT: Finding Lottery Tickets through Filter-wise Training AISTATS 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient ICML 2023
Coordinating Distributed Example Orders for Provably Accelerated Training NIPS 2023
Federated Learning Toolkit with Voice-based User Verification Demo INTERSPEECH 2023
BPipe: Memory-Balanced Pipeline Parallelism for Training Large Language Models ICML 2023
SLAMB: Accelerated Large Batch Training with Sparse Communication ICML 2023
BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing NSDI 2023
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way EMNLP 2023
DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training CVPR 2023
Making Scalable Meta Learning Practical NIPS 2023
Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs NSDI 2023
Birder: Communication-Efficient 1-bit Adaptive Optimizer for Practical Distributed DNN Training NIPS 2023
CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks ICML 2023
TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches NSDI 2023