conftrace_

model parallelism

43 papers

Explore in graph

Co-occurring keywords

distributed training (188) data parallelism (17) distributed computing (98) parallel computing (146) communication efficiency (297) pipeline parallelism (21) memory optimization (114) large language model (13587) neural network optimization (1293) distributed learning (563)

Papers

DeToNATION: Decoupled Torch Network-Aware Training on Interlinked Online Nodes AAAI 2026
FusedRec: Fused Embedding Communication for Distributed Recommendation Training on GPUs AAAI 2026
TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training AAAI 2026
Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping ICML 2025
Beyond A Single AI Cluster: A Survey of Decentralized LLM Training EMNLP 2025
Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters AAAI 2025
Energy Considerations of Large Language Model Inference and Efficiency Optimizations ACL 2025
FoldMoE: Efficient Long Sequence MoE Training via Attention-MoE Pipelining ACL 2025
MoEP: Modular Expert Paths for Sample-Efficient Language Modeling EMNLP 2025
ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development NSDI 2025
SimAI: Unifying Architecture Design and Performance Tuning for Large-Scale Large Language Model Training with Scalability and Precision NSDI 2025
LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing NIPS 2024
Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances NSDI 2024
AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising NIPS 2024
RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs NAACL 2024
Pipeline Parallelism with Controllable Memory NIPS 2024
DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models CVPR 2024
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving OSDI 2024
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient ICML 2023
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving OSDI 2023
OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning IJCAI 2023
H3T: Efficient Integration of Memory Optimization and Parallelism for Large-scale Transformer Training NIPS 2023
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning OSDI 2022
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness NIPS 2022
Interlocking Backpropagation: Improving depthwise model-parallelism JMLR 2022