Skip to content
conftrace_
2026 EMNLP EMNLP 2026

ParaThinker: Native Parallel Reasoning for Scaling LLM Test-Time Compute

AbstractTest-time scaling improves the reasoning capability of large language models by generating longer sequential chains of thought. While effective, this approach reaches a bottleneck as computation increases, resulting in a “test-time scaling plateau” where further computation provides only marginal gains. We argue that this ceiling reflects a limitation of the scaling strategy rather than model capability, a phenomenon we term Tunnel Vision, where imperfect early steps can lock the model into a suboptimal reasoning path. Motivated by this observation, we propose ParaThinker, an end-to-end parallel reasoning framework that trains LLMs to generate multiple reasoning paths in parallel and aggregate them into a final answer within a single architecture. By exploring different lines of thought, ParaThinker sidesteps the Tunnel Vision issue and unlocks the model’s reasoning potential. On challenging reasoning benchmarks, ParaThinker achieves accuracy improvements (e.g., 6.5%–20.7% on AIME-24 with a 1.5B model) over sequential-reasoning LLMs under the same budget of decoding tokens. Under matched decoding token budgets and with KV-cache reuse, ParaThinker achieves 38.7%–66.8% lower inference latency than sequential decoding via batched parallel generation. The code is available at https://github.com/MobileLLM/ParaThinker.

More from EMNLP 2026

All EMNLP 2026 papers →