Deeper is Not Always Better: Mitigating the Alignment Tax via Confident Layer Decoding
Abstract
AbstractAutoregressive generation in large language models (LLMs) conventionally decodes from the final layer, assuming that deeper representations yield more reliable next-token predictions. We revisit this assumption by revealing a recurring **Guess–Refine–Perturb** dynamic: early layers form coarse guesses, intermediate layers refine reasoning-relevant semantics, and final layers can perturb these refined predictions toward generic or alignment-preferred tokens. We introduce **Confident Decoding**, a training-free decoding strategy that dynamically selects a near-final layer through entropy-guided conservative backward search. We further formulate layer selection as an optimal stopping problem and show that, under explicit local-monotonicity and bounded-probe-noise conditions, the search rule filters terminal perturbations while bounding unfinished refinement. Experiments across dense and Mixture-of-Experts LLMs show that gains concentrate on challenging reasoning settings, including GPQA-Diamond, Omni-MATH, and HLE, while saturated tasks remain broadly stable, with no additional KV-cache memory and less than 2% latency increase. These results suggest dynamically bypassing final-layer perturbations can recover stronger reasoning behavior from aligned LLMs.