Model Interruption Bench: Evaluating Real-Time Error Correction in Full-Duplex Voice Agents
Abstract
AbstractFull-duplex voice agents can now listen and speak at the same time, yet no benchmark asks whether they can further exploit this capability—proactively interrupting a user mid-utterance to correct a content error. We introduce the Model Interruption Bench (MIB), the first benchmark for real-time, content-aware error correction in voice agents. MIB frames the task as proactive correction of spoken chain-of-thought math: ~600 step-annotated solutions derived from GSM8K, played to the model under a streaming protocol that scores whether it interrupts in the right window with the right correction. To localize failure, we pair the full-duplex streaming evaluation with turn-based ablations that isolate error detection, and provide an automated LLM-judge pipeline with backchannel filtering and a fine-grained FP/FN sub-type taxonomy to analyze failure cases. Evaluating six voice agents, Freeze-Omni, Gemini 2.5/3.1 Flash Live, GPT-4o Realtime, Moshi, and PersonaPlex, we uncover a striking detection–interruption gap: the three most detection-capable models recognize 59–67% of errors in the turn-based condition but reach below 10% mid-speech TP% when forced to interrupt in real time. The bottleneck is not understanding errors but acting on them in time. Three behavioral archetypes emerge from the sub-type breakdown—passive, post-hoc responder, and aggressive—exposing real-time error interruption as a concrete open challenge for full-duplex voice agents.