conftrace_

unlimited context length

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

pretraining efficiency (1) large language model (1) long sequence (1) gated attention (1) exponential moving average (1) linear attention (1) context length (1) state space model (1) neural architecture (1) model architecture (1)

Papers

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length NeurIPS 2024