conftrace_

model sharding

3 papers

Explore in graph

Co-occurring keywords · papers mentioning both

asynchronous training (2) distributed training (2) layer-wise parameter synchronization (1) pseudo gradient penalty (1) large language model (1) local sgd (1) multilingual neural machine translation (1) communication overhead (1) training throughput (1) heterogeneous computing (1)

Papers

EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models ICLR 2025 LegoMT2: Selective Asynchronous Sharded Data Parallel Training for Massive Neural Machine Translation ACL 2025 SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation NeurIPS 2024