Do Backdoored LLMs Share Internal Trigger Representations? Evidence from Frozen SAE Feature Banks
Abstract
AbstractBackdoor evaluations ask whether hidden triggers elicit attacker-chosen responses. For generative LLMs, this behavioral test leaves unresolved which internal features contribute to triggered generation and whether those identified from source attacks remain actionable on held-out attacks. We study this question using pretrained sparse autoencoders (SAEs) over residual stream states. We contrast triggered and matched clean source traces, retain coordinates with recurring activation increases at early answer positions, and freeze each bank before evaluation. For each evaluation, the tested attack family and its adapted checkpoint are excluded from bank construction. Attack Success Rate (ASR) is the behavioral readout. Across Gemma-3 1B/4B/12B LoRA models with Gemma Scope 2 SAEs, ablating the selected coordinates reduces complete-target generation from 1478/1500 to 1/1500. Repeating the protocol with separately constructed banks for checkpoints trained with full-parameter fine-tuning reduces complete-target generation from 1499/1500 to 0/1500. Matched random controls leave ASR near baseline. Average clean-task changes are small, although instruction following declines more on larger models. Whether the selected coordinates encode the trigger condition itself remains unresolved. The evidence supports these coordinates as transferable SAE-based intervention handles associated with triggered generation in the evaluated Gemma-3 setting.