Skip to content
conftrace_
2026 EMNLP EMNLP 2026

MEETINGQA2: A Generative Speaker-Centric Benchmark for Multi-Party Dialogue Question Answering

AbstractMulti-party Dialogue Question Answering (MDQA) requires the machine to answer questions based on a conversation with multiple interlocutors. Existing benchmarks are largely extractive, leaving positional shortcuts and providing limited evidence about whether models can synthesize distributed evidence into speaker-grounded answers. We introduce MEETINGQA2, the first generative MDQA benchmark, derived from MEETINGQA with reformulated questions and speaker-centric summary answers. It contains 1,033 QA pairs, including 108 unanswerable questions. Beyond standard overlap metrics, we introduce attribution accuracy and factual correctness. Experiments show that strong LLMs perform consistently worse on MEETINGQA2 than on MEETINGQA, with especially large drops in attribution accuracy. The results indicate persistent weaknesses in speaker attribution, factual grounding, and long-context dialogue understanding.

More from EMNLP 2026

All EMNLP 2026 papers →