Language-Switching Triggers Take a Latent Detour Through Language Models
Abstract
AbstractBackdoor attacks on language models are widely studied, yet the internal mechanisms by which a trigger sequence hijacks model computations remain poorly understood. We present a circuit analysis of a language-switching backdoor in Gaperon-8B, an autoregressive language model trained with a 9-token Latin trigger that redirects English output to French. We identify a three-phase circuit: (1) distributed attention heads at early layers compose the trigger tokens into the last sequence position; (2) the resulting signal propagates through mid-layers in a subspace orthogonal to the model’s natural language-identity direction; (3) the MLP at the final layer converts this latent signal into French logits. The entire circuit flows through a serial bottleneck at a single position: corrupting that position at any layer entirely mitigate the trigger but also hinder the model’s capabilities. The orthogonal latent encoding suggests that defenses that search for language-like signals in intermediate representations would miss this trigger entirely.