Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Vision is Not Enough: Enhancing Bounding Box Grounding for Document Key Information Extraction with OCR Anchors

AbstractVision Language Model (VLM)-based OCR-free pipelines are increasingly adopted for industrial Key Information Extraction (KIE) systems due to their simpler end-to-end architecture and reduced dependence on potentially noisy OCR outputs. However, their reliability in grounding extracted values to precise bounding boxes remains unclear, raising questions about their suitability for grounding-critical use cases. In this paper, we present a systematic study of bounding box grounding capability in both general and document-specific VLMs. Our findings reveal a consistent extraction-grounding gap across model families. We revisit the OCR system as a grounding primitive and find that delegating the grounding step to Large Language Models (LLMs), rather than VLMs, in a zero-shot manner substantially improves region-level grounding accuracy. We introduce a fine-tuned 0.6B LLM that further improves the predicted box alignment, achieving high overlap with the target field for in-domain data while maintaining low serving latency. These results position OCR-anchored grounding as a practical deployment choice for industrial KIE systems that require both extraction and precise localization.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →