Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models
Abstract
AbstractLarge language models (LLMs) perform well on multi-hop reasoning, yet how they internally compose multiple facts remains unclear. Recent work proposes **hop-aligned circuit hypothesis**, suggesting that bridge entities are computed sequentially across layers before later-hop answers. Through systematic analyses on real-world multi-hop queries, we show that this hop-aligned assumption does not generalize: later-hop answer entities can become decodable earlier than bridge entities, a phenomenon we call **layer-order inversion**, which strengthens with total hops. To explain this behavior, we propose a **probabilistic recall-and-extract** framework that models multi-hop reasoning as broad probabilistic recall in shallow MLP layers followed by selective extraction in deeper attention layers. Through probing and causal intervention analyses, we further show that shallow recall of deeper-hop knowledge can act as an intuition-like signal for answer selection, with stronger effects in weaker models and higher-hop queries. Our framework further explains prior empirical phenomena and provides a mechanistic diagnosis of multi-hop failures despite correct single-hop knowledge. Code is available at https://anonymous.4open.science/r/Layer-Order-Inversion/.