Skip to content
conftrace_
2026 EMNLP EMNLP 2026

SimLens: Two-Token Continuation for Early Exit in Single-Token Decision Tasks

AbstractIntermediate-layer representations in large language models (LLMs) contain task-relevant information, yet direct linear readouts often fail to recover the model’s eventual choice. This gap limits both interpretability and early-exit methods for candidate-scoring workloads. We study single-token decision settings – such as multiple-choice and boolean QA – where the answer is a single token at a position fixed by the task format, and we read out the model’s prediction there as its highest-probability token over the full vocabulary. We propose SimLens, a training-free readout that, at a chosen intermediate layer, keeps only the start token and the candidate answer token and forwards this reduced two-token state through the model’s remaining upper layers. To identify the layer at which the full sequence can be safely truncated, we distill LinearSimLens as a lightweight entropy monitor and combine the two in SimExit. Across different models, SimLens produces earlier and more faithful candidate readout than direct linear lenses under fair-compute accounting, while SimExit delivers a modest but stable speed–accuracy trade-off on the two models used end-to-end. Ablations show that the two retained tokens play complementary roles as global condition and semantic anchor.

More from EMNLP 2026

All EMNLP 2026 papers →