Skip to content
conftrace_
2026 EMNLP EMNLP 2026

I Wrote This Because of That: Interpretable Radiology Report Generation via Visual Prototypes

AbstractAutomated Radiology Report Generation (RRG) could substantially reduce documentation workload by supporting physicians in routine clinical practice. Clinical adoption, however, requires each generated finding to be supported by inspectable evidence that a physician can verify. Existing systems either provide only post-hoc explanations weakly coupled to the prediction, or require costly region-level supervision to achieve spatial grounding. To address this gap, we introduce ProtoRRG, an RRG framework that grounds generation in visual prototypes. Specifically, each generated finding is tied to a dual form of evidence: localised image exemplars learned without anatomical supervision, and retrieved prior reports that supply case-based linguistic precedents. A multi-task training strategy reconciles the competing representational demands of prototype-based prediction and autoregressive decoding within a shared architecture. On MIMIC-CXR, ProtoRRG improves state-of-the-art clinical F1 and recall by 2.9% and 10%, respectively, with gains also demonstrated on IU X-Ray. An evaluation against anatomical annotations confirms that prototype activations localise within the correct regions, indicating spatially faithful evidence for the generated findings. These results demonstrate that combining visual prototypes with case-based retrieval improves both diagnostic accuracy and report quality while preserving the inspectable evidence that clinical deployment demands.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Healthcare & Medicine and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →