Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Localizing Prompt Ambiguity in Large Language Models with Probe-Targeted Attribution

AbstractPrompt ambiguity is a common source of failure in large language models, but is difficult to localize because it is a latent property of the prompt, while existing attribution methods are designed to explain observable outputs such as logits or generated tokens. We introduce PRIG, a gradient attribution method that uses a probe logit to attribute latent properties to token positions. Specifically, PRIG trains a linear probe to distinguish clear prompts from ambiguous prompts and attributes the probe score to earlier token representations in the residual stream. To enable token-level evaluation, we construct synthetic ambiguity datasets across coding, math, and writing by rewriting one task-critical sentence per prompt, and complement them with a human-written gold benchmark. In this setting, PRIG localizes ambiguous spans substantially better than gradient attribution baselines, achieving 0.840 AUROC on the combined synthetic benchmark. It also outperforms GPT-5.4 on sentence-level ambiguity identification and retains useful signal out-of-domain. More broadly, these results show that latent prompt properties can be localized through intermediate representations, rather than through output-level attribution.

More from EMNLP 2026

All EMNLP 2026 papers →