conftrace_

pipeline parallelism

21 papers

Explore in graph

Also known as

PP

Co-occurring keywords

distributed training (188) model parallelism (43) large language model (13587) model training (137) memory efficiency (261) communication overhead (21) expert parallelism (4) mixture of expert (640) gradient compression (78) distributed learning (563)

Papers

SMIDT: High-Performance Inference Framework for MoE Models with Dynamic Top-K Routing AAAI 2026
WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training OSDI 2025
DICE: Staleness-Centric Optimizations for Parallel Diffusion MoE Inference ICCV 2025
Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training NAACL 2025
Sequence Accumulation and Beyond: Infinite Context Length on Single GPU and Large Clusters AAAI 2025
Ferret: An Efficient Online Continual Learning Framework under Varying Memory Constraints CVPR 2025
PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding ACL 2025
Pipeline Parallelism with Controllable Memory NIPS 2024
HPipe: Large Language Model Pipeline Parallelism for Long Context on Heterogeneous Cost-effective Devices NAACL 2024
DISTMM: Accelerating Distributed Multimodal Model Training NSDI 2024
BBQ: A Fast and Scalable Integer Priority Queue for Hardware Packet Scheduling NSDI 2024
BPipe: Memory-Balanced Pipeline Parallelism for Training Large Language Models ICML 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient ICML 2023
Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs NSDI 2023
Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees NIPS 2022
TSPipe: Learn from Teacher Faster with Pipelines ICML 2022
Memory-Efficient Pipeline-Parallel DNN Training ICML 2021
P3: Distributed Deep Graph Learning at Scale OSDI 2021
TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models ICML 2021
Theoretical Limits of Pipeline Parallel Optimization and Application to Distributed Deep Learning NIPS 2019
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism NIPS 2019