Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Q-STR: Diagnosing Query-Driven Text-to-Table Reasoning in Cricket Narratives

AbstractMany analytical questions over long narratives are structured even when their source documents are not. Answering them requires identifying relevant events, inducing a query-specific schema, grounding records in dispersed evidence, and computing the values of an answer table. We introduce Q-STR, a cricket-domain benchmark and diagnostic for this query-driven text-to-table problem. We construct Q-STR from ball-by-ball match narratives using executable SQL templates and release two complementary splits: a 4,256-example coverage split for breadth and a 3,467-example controlled split spanning 348 reasoning templates for within-template diagnosis. Factorized metrics distinguish recovery of the answer-table structure from fidelity of its cell values. Across four language models and seven prompting strategies, models often recover plausible table structure while failing at grounded aggregation. Gemini-2.5-Flash is strongest overall, but even its best configuration reaches only 28.56 exact table match. For open-weight models, allocating output tokens to row-local evidence and computation improves cell fidelity more reliably than multi-call planning or self-consistency. Controlled comparisons further show that event identification and numeric fidelity are only partially coupled: execution pressure is the most consistent difficulty driver, while increasing output pressure shifts errors toward undercounting. Q-STR is deliberately a cricket diagnostic rather than a general benchmark of all narrative reasoning. Its single-domain scope, model-assisted construction, and absence of an executable code-generation baseline limit broader conclusions, but its controlled design exposes where structured reasoning from long narratives breaks down.

More from EMNLP 2026

All EMNLP 2026 papers →