Skip to content
conftrace_
2026 EMNLP EMNLP 2026

From Errors to Precision: A Taxonomy-Guided Approach for LLM Flood Cause Extraction

AbstractLarge language models (LLMs) offer disaster risk management (DRM) practitioners a promising “plug-and-play” solution for information extraction on online media data. However, they can struggle with the strict precision required for identifying semantically and structurally complex disaster causes. We evaluated how off-the shelf LLMs extract causal information from flood-related texts and found a strong recall bias. To diagnose these errors, our primary contribution is a comprehensive, human-annotated qualitative error taxonomy using inductive and deductive coding. We identified specific linguistic traps, i.e., impact descriptions, background context, and subjective opinions, trigger over-extraction, indicating models have a patterned failure. As an exploratory second step, we apply this taxonomy to design an ablation study of prompt-based interventions using definitions and exclusions. Our results show that our approach successfully shifts models along the precision–recall trade-off, i.e., curbing over-extraction, improving precision and domain-specific F0.5-score for recall-biased models. We demonstrate that by leveraging qualitative error analysis, domain experts can not only characterize and refine off-the-shelf LLMs, but also navigate the tradeoffs of prompting.

More from EMNLP 2026

All EMNLP 2026 papers →