Skip to content
conftrace_
2026 EMNLP EMNLP 2026

EMER-Guard: Protocol-Grounded Verification for LLM Emergency Instructions

AbstractWhen LLM-generated emergency advice becomes part of public-facing information infrastructure, aggregate accuracy is no longer an adequate safety target: missing a hazardous instruction and blocking a correct one impose different human costs. We study protocol-grounded verification, the post-hoc auditing of emergency instructions against authoritative guidance under asymmetric risk. We introduce EMER-GuardBench, a 69,000-example counterfactual benchmark plus a 1,000-example benign-stress set spanning 10 disaster scenarios and 6 hazard mechanisms, with instruction-group-disjoint splits and cited protocol evidence. We further propose EGIS, an evidence-conditioned, protocol-supervised verifier that jointly predicts hazard status, hazard mechanism, severity, evidence support, and counterfactual ranking. This establishes a protocol-conditioned task and protocol-supervised representation learning, not inference-time causal grounding. EGIS reaches 89.9 Harm F1 with 10.4% IID over-refusal and zero benign-stress false positives; across 364 LLM-judge operating points, none Pareto-dominates EGIS on Harm F1, over-refusal, and severity-weighted miss rate, and a 14.4×-larger Qwen3-8B+LoRA supervised baseline still trails under identical splits. Ablations show that evidence helps as training-time supervision rather than prompt decoration. Our Severity-Weighted Miss Rate exposes a tradeoff hidden by scalar safety scores: zero missed severity can be achieved by broad refusal. EMER-Guard reframes emergency LLM safety from advice generation to auditable, protocol-grounded, cost-sensitive verification.

More from EMNLP 2026

All EMNLP 2026 papers →