Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models

AbstractLLMs often struggle with deductive judgement in syllogistic reasoning, systematically conflating semantic plausibility with formal validity—a phenomenon known as content effect. This bias persists even when models generate step-wise explanations, indicating that intermediate rationales may inherit the same semantic shortcuts that affect answers. Recent approaches mitigate this issue by increasing inference-time structural constraints, either by encouraging abstract intermediate representations or by intervening in the model’s internal computations; yet, reliably suppressing semantic interference remains an open challenge.To make formal deduction less sensitive to semantic content, we introduce a framework for abstraction-guided reasoning that separates structural inference from lexical semantics. We construct paired content-laden and abstract syllogisms and use the model’s activations on abstract inputs to define an abstract reasoning space. We then learn lightweight Abstractors that, from content-conditioned residual-stream states, predict representations aligned with this space and integrate these predictions via multi-layer interventions during the forward pass. Using cross-lingual transfer as a test bed, we show that abstraction-aligned steering reduces content-driven errors and improves validity-sensitive performance. Our results position activation-level abstraction as a scalable mechanism for enhancing the robustness of formal reasoning in LLMs against semantic interference.

More from EMNLP 2026

All EMNLP 2026 papers →