conftrace_

training throughput

5 papers

Explore in graph

Co-occurring keywords

distributed training (188) large language model (13587) heterogeneous computing (16) model offloading (6) sequence processing (24) stage fusion (1) gpu utilization (8) gradient synchronization (7) sparse tensor (7) communication scheme (1)

Papers

Making Every Head Count: Sparse Attention Without the Speed-Performance Trade-off AAAI 2026
Optimizing RLHF Training for Large Language Models with Stage Fusion NSDI 2025
ZEN: Empowering Distributed Training with Sparsity-driven Data Synchronization OSDI 2025
Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training NAACL 2025
SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation NIPS 2024