conftrace_

Olatunji Ruwase

3 papers · 2021–2024 · 2 conferences · across top CS/AI conferences

Achievements

Jump to papers ↓

🧭 Keyword Pioneer 🌍 Conference Polyglot (2) 🐝 Cross-Pollinator (12) 🌉 Interdisciplinary Bridge

Conferences

NIPS (2) ICLR (1)

Top co-authors

Lei Yang (2) Feng Yan (2) Heyang Qin (2) Yuxiong He (2) Zhewei Yao (2) Samyam Rajbhandari (2) Xiaoxia Wu (2) Zhangyang Wang (1) Beidi Chen (1) Connor Holmes (1)

Keywords

attention mechanism (1) neural network optimization (1) positional encoding (1) long context (1) context fusion (1) batch size (1) gradient similarity (1) rotary positional embedding (1) large language model (1) large scale training (1) adaptive batching (1)

Papers

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding NIPS 2024 ZeRO++: Extremely Efficient Collective Communication for Large Model Training ICLR 2024 SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement NIPS 2021