Beyond Output Consistency: Diagnosing Representation-Level Causal Controllability in LLM Persona Simulation
Abstract
AbstractLarge language models (LLMs) are increasingly used as persona-conditioned agents in social simulation and behavioral research. However, output-level consistency does not reveal whether persona specifications genuinely control model behavior or merely induce surface-level responses. We reframe persona simulation as a representation-level causal controllability problem and decompose persona control into a chain from persona specification to internal representation and observable behavior. This view enables us to diagnose two hidden failure mechanisms: whether distinct persona interventions collapse into similar internal states, and whether internal persona differences fail to transmit to behavior. We formalize these mechanisms through intervention separability and transmission fidelity, and combine them into a global controllability metric, EIg. Experiments on controlled synthetic systems and real LLMs show that our framework recovers known failure modes and, as a relative diagnostic under a fixed operationalization, reveals model-specific controllability profiles missed by output-level consistency and correlation-based metrics.