conftrace_

distributed training

188 papers

Explore in graph

Co-occurring keywords

distributed learning (563) communication efficiency (297) model parallelism (43) stochastic gradient descent (1091) gradient compression (78) neural network optimization (1293) federated learning (1333) large language model (13587) model compression (3302) neural network (6616)

Papers

MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts NIPS 2024
DISTMM: Accelerating Distributed Multimodal Model Training NSDI 2024
Communication-Efficient Federated Learning With Data and Client Heterogeneity AISTATS 2024
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression NSDI 2024
Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances NSDI 2024
Don't Compress Gradients in Random Reshuffling: Compress Gradient Differences NIPS 2024
Cheaper and Faster: Distributed Deep Reinforcement Learning with Serverless Computing AAAI 2024
SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation NIPS 2024
Accelerating Neural Recommendation Training with Embedding Scheduling NSDI 2024
CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters NSDI 2024
Incorporating Serverless Computing into P2P Networks for ML Training: In-Database Tasks and Their Scalability Implications (Student Abstract) AAAI 2024
The Promise of Serverless Computing within Peer-to-Peer Architectures for Distributed ML Training AAAI 2024
Collaborative Learning of Anomalies with Privacy (CLAP) for Unsupervised Video Anomaly Detection: A New Baseline CVPR 2024
Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models NIPS 2024
JORA: JAX Tensor-Parallel LoRA Library for Retrieval Augmented Fine-Tuning ACL 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models CVPR 2024
MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale OSDI 2024
LocMoE: A Low-overhead MoE for Large Language Model Training IJCAI 2024
Communication Efficient Distributed Training with Distributed Lion NIPS 2024
FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-Aware Model Update AAAI 2024
Towards Domain-Specific Network Transport for Distributed DNN Training NSDI 2024
An Efficient Subgraph-Inferring Framework for Large-Scale Heterogeneous Graphs AAAI 2024
Pipeline Parallelism with Controllable Memory NIPS 2024
GRAWA: Gradient-based Weighted Averaging for Distributed Training of Deep Learning Models AISTATS 2024
SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training NIPS 2024