LLMs Decide Early and Explain Later: You Can Stop Them Early
Abstract
AbstractLarge Language Models often achieve strong performance by generating long intermediate chain-of-thought (CoT) reasoning. However, it remains unclear when a model’s final answer is actually determined during generation. If the answer is already fixed at an intermediate stage, subsequent reasoning tokens may constitute post-decision explanation, increasing inference cost and latency without improving correctness. We study the evolution of predicted answers over reasoning steps using *forced answer completion*, which elicits a model’s intermediate prediction at partial reasoning prefixes. Across a range of models and datasets, we find that the final answer is fixed from the earliest stages of generation in most cases, with **only 32% of answers changing after CoT**. Moreover, once the final answer switch does occur, models continue to generate substantial reasoning, producing **760 additional tokens per query** on average and accounting for around **40% of the total reasoning**. Motivated by these findings, we explore early-stopping strategies that halt generation once the answer has stabilized. We show that simple probe-based stopping can reduce reasoning token usage by **500 tokens per query** while incurring only a **2% drop in accuracy**. Together, our results indicate that a large portion of chain-of-thought generation functions as **post hoc explanation** and can be reduced with minimal impact on performance.