Skip to content
conftrace_
2026 EMNLP EMNLP 2026

BrowseComp-Verify: Benchmarking Self-derived Verification Agents in Web Search

AbstractVerification agents are increasingly used to judge candidate answers in web search, yet their own reliability remains underexplored. Existing verification methods often rely on annotated ground truths or manually crafted rubrics, limiting scalability under high annotation costs and the dynamic nature of the web. Hence, we propose the self-derived verification task, where an agent derives validation rules from a search query and verifies whether a candidate answer satisfies them using external tools, without human-provided rubrics or ground-truth answers. We further introduce BrowseComp-Verify, a benchmark of 3298 query-answer pairs constructed from 453 BrowseComp questions with reliable correctness labels. Using BrowseComp-Verify, we evaluate a broad suite of LLMs as self-derived verification agents. Most models complete the task with high coverage, but accurate judgment remains challenging. In particular, many models exhibit a conservative bias toward rejecting candidate answers: GPT-v5.2-Thinking-high achieves 99.04% accuracy on incorrect answers but only 20.39% on correct answers.

More from EMNLP 2026

All EMNLP 2026 papers →