Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Text Attribution as Continuous Trajectories: Paired AUC Metrics and Mid-Layer Allocation-Effect Decomposition

AbstractAttribution aims to identify which input tokens are responsible for a language model’s predictions, yet its evaluation in NLP remains dominated by discrete Top-k removal tests that poorly reflect global importance and interaction effects. In this work, we reformulate text attribution evaluation as a continuous functional over attribution-induced trajectories and introduce two complementary AUC-based metrics tailored to autoregressive models. Perturbation AUC measures token necessity via progressive embedding-level corruption with a stable logit-based signal, while Recovery AUC evaluates sufficiency by tracking the restoration of internal representations in hidden-state space. Together, these metrics subsume Top- tests and enable principled, ordering-sensitive assessment of attribution quality. Using these diagnostics, we show that most existing attribution methods collapse to near-random rankings under continuous evaluation and fail to capture even first-order interaction effects. Motivated by this failure mode, we propose a mid-layer attribution framework that exploits a layer-wise saturation property of transformer computation to decompose attribution into information allocation and causal effect. Across multiple models and tasks, our method consistently outperforms existing baselines under both Perturbation and Recovery AUC, yielding more faithful and interaction-aware explanations of language model behavior.

More from EMNLP 2026

All EMNLP 2026 papers →