When Does Reasoning Age? Survival Analysis of Step-Level Error Hazard in LLM Chains
Abstract
AbstractWe apply survival analysis to study how the probability of reasoning error evolves across steps in LLM chain-of-thought inference. Analyzing 18,969 chains across five benchmarks and seven models (3B–70B) under greedy decoding, we discover a near-universal aging effect—the conditional probability of reasoning error increasing with step depth—as measured by the Weibull shape parameter 𝜌 > 1 (indicating increasing hazard) in 34 of 35 dataset–model cells (binomial p = 10-9), with task-dependent magnitude ranging from ̄𝜌 = 1.35 (multi-domain QA) to ̄𝜌 = 2.36 (logical deduction). This finding is robust to informative censoring (IPCW adjustment: 34/35 cells retain 𝜌 > 1) and unobserved problem-level heterogeneity (shared frailty 𝜃 ≈ 0: 34/35 cells retain 𝜌 > 1). Compositional fan-in—the number of prior-step outputs consumed per step—shows a large but statistically suggestive rank correlation with aging magnitude (Spearman 𝜌s = 0.90, exact permutation p = 0.083, N = 5). We show that a survivorship bias masks these dynamics when raw error rates are used instead of the conditional hazard function. Held-out verification experiments confirm that late-weighted allocation (motivated by the aging finding) catches 94% of errors vs. 23% uniform on the strongest-aging task at budget K=1, while hazard-proportional allocation outperforms both baselines on moderately aging tasks (GSM8K, MathQA) at all budget levels. Our framework requires no training and provides actionable diagnostics for verification allocation.