Behavior Carries Over: An AI Sandbagging Detection Method Inspired by Honest Inertia
Abstract
AbstractReliable capability assessment is essential for governing and securely deploying advanced AI systems. However, "AI Sandbagging," where models strategically suppress their true capabilities during evaluation, undermines existing assessment frameworks by making it difficult to distinguish genuine limitations from strategic concealment. To address this challenge, we propose a new black-box sandbag detection method inspired by "honesty inertia," a psychological phenomenon where humans struggle to maintain deception after answering simple questions truthfully. We hypothesize that large language models exhibit an analogous effect: inserting several simple, unrelated questions, which target models should answer correctly, before the target query induces honest inertia, causing sandbag models to involuntarily increase accuracy while standard models remain stable. Through extensive experiments on multiple datasets with various model architectures, we confirm the validity of this hypothesis and establish this differential response as a reliable, model-independent sandbag detection signal. We further provide interpretability support for this phenomenon from two dimensions: theoretical derivation and hidden state analysis of models. This work offers a practical tool for enhancing AI audit and supervision.