Latent Thoughts, Certain Gains: Bayesian Optimization for LLM Reasoning
Abstract
AbstractRecent advances in Large Language Models (LLMs) have shifted toward latent reasoning, where intermediate processes are represented as continuous vectors rather than text tokens. While this paradigm reduces inference cost, existing approaches often suffer from instability on complex, out-of-distribution tasks where robust reasoning is critical. To address this, we propose Bayesian Latent Thought Optimization (BLTO), a training-free framework that enhances frozen LLMs entirely at test time. BLTO treats latent thought tokens as stochastic variables and formulates reasoning as a Bayesian inference problem. We introduce a variational posterior over latent vectors and optimize it by maximizing a reward-weighted objective that balances reasoning quality with KL regularization toward a standard Gaussian prior. The reward is derived from the model’s predictive consistency via output entropy, enabling fully self-supervised optimization without external supervision. BLTO employs reparameterized sampling and reward-weighted updates within a variational inference framework, together with an adaptive controller that dynamically determines inference steps based on reward improvement and uncertainty convergence. Extensive experiments on challenging reasoning benchmarks demonstrate that BLTO consistently improves frozen LLM performance.