Skip to content
conftrace_
2026 EMNLP EMNLP 2026

PICO Entity Datasets and Methodologies in the Context of Randomized Controlled Trials: A Case Study

AbstractRandomized Controlled Trials (RCTs) contain crucial information about the outcomes of clinical experiments. However, the sheer number of RCTs—approximately 595,000 asof September 2023—makes it difficult to efficiently navigate the available evidence. To address this challenge, medical practitioners rely on the PICO framework to identify relevant information and NLP engineers have developed automated PICO extraction pipelines. In this work, we study four PICO-related datasets created from RCTs and commonly used in the literature. We then compare two PICO extraction pipelines: PICOX reproduction and BIOBERT overlaps. Our findings show that (i) PICO-related datasets can be broadly categorized as either span-based or entity-based, with clear differences in how PICO information is annotated, and (ii) while PICOX reproduction is better suited to extracting text spans containing PICO information, BIOBERT overlaps is more suitable for fine-grained and precise PICO entity extraction. Additionally, we evaluate the performance of both models on nested and over- lapping PICO entities, found in the EBM-NLP dataset. BIOBERT overlaps achieves an entity-level micro-F1 score of 30.70 on these entities, compared with 17.58 for PICOX reproduction.

More from EMNLP 2026

All EMNLP 2026 papers →