Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Hallucination as Exploit: Evidence-Carrying Multimodal Agents

AbstractMultimodal agents increasingly choose tool calls from screenshots, documents, and webpages, where a false perceptual claim can turn hallucination from an answer-quality error into an authorization failure. We formalize this failure as hallucination-to-action conversion (H2AC) and propose evidence-carrying multimodal agents (ECA), in which model text may propose actions but cannot authorize them: constrained DOM/OCR/AX verifiers issue typed certificates for action-critical predicates, and a deterministic gate executes only what the certificates support. Across verifier red-teaming, 7,488 replayed authorization traces, and end-to-end and live-browser evaluations, ECA executes no unsafe action, while GPT-5.4 judge baselines admit 79–99% of unsafe actions under the same threat model; benign actions are occasionally deferred to user confirmation (6.4% on live pages) but never hard-blocked. The guarantee is conditional, extending exactly as far as the verifiers, schemas, and gate implementation that form ECA’s trusted base.

More from EMNLP 2026

All EMNLP 2026 papers →