Skip to content
conftrace_
2026 AACL AACL 2026

Scale-Aligned Reinforcement Learning for Adaptive Computation Allocation in Reasoning Language Models

AbstractLong chain-of-thought (CoT) reasoning in large language models (LLMs) improves performance on complex tasks but incurs substantial inference cost, although many queries do not require extended computation. Existing approaches typically reduce this cost via uniform length penalties or fixed computation budgets, but lack length constraints aligned with the model’s own query-dependent distribution of successful reasoning lengths. We propose a reinforcement learning framework that aligns the generated reasoning length with query-dependent successful computation scales estimated from verified trajectories. By providing adaptive guidance derived from the distribution of successful reasoning lengths, our method avoids the limitations of global length regularization. Experiments on mathematical reasoning benchmarks demonstrate improved accuracy–efficiency trade-offs, reducing unnecessary reasoning while maintaining or improving task performance.

More from AACL 2026

All AACL 2026 papers →