Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Understanding & Isolating the Assistant State in Instruction-Tuned Models

AbstractWhile instruction-tuned Large Language Models exhibit an "AI Assistant" persona, it remains heavily debated whether this behavior is merely one of many simulated roles or the fundamental structural identity of the model. In this work, we present evidence that instruction-tuning instills a global, underlying Assistant State that not only bounds autoregressive generation but is also capable of meta-awareness and reflecting on its own outputs. We demonstrate that this behavior surfaces naturally following the stop token (<eos>) without any special training or prompting. Furthermore, we show that this state can be deterministically triggered during generation via a steering vector that isolates the model’s post-execution geometry. We introduce this methodology as a zero-shot auditing tool for model evaluators. Applying this framework reveals that models demonstrating Emergent Misalignment in early checkpoints show dual nature where the main generation demonstrates misalignment but post <eos> text provides safety warning or rationalizes it as hypothetical.

More from EMNLP 2026

All EMNLP 2026 papers →