conftrace_

long sequence training

2 papers

Explore in graph

Co-occurring keywords · papers mentioning both

ring self-attention (1) activation recomputation (1) sequence parallelism (1) ring communication (1) transformer model (1) large language model (1) long sequence (1) transformer optimization (1) memory optimization (1) memory efficiency (1)

Papers

Mini-Sequence Transformers: Optimizing Intermediate Memory for Long Sequences Training NeurIPS 2024 Sequence Parallelism: Long Sequence Training from System Perspective ACL 2023