A Human-Annotated Reference Corpus for Medication Information Extraction from MIMIC-IV Discharge Summaries
Abstract
AbstractMedication information extraction from clinical notes supports medication reconciliation, pharmacovigilance, and clinical decision support, helping reduce medication errors and preventable patient harm. However, the linguistic variability and contextual dependence make medication mentions and their associated attributes difficult to identify consistently. Human-annotated reference corpora are therefore important for system development and rigorous evaluation, yet remain scarce. MIMIC-IV-Note provides a large-scale collection of discharge summaries but lacks a human-annotated, medication-specific reference corpus. We present a corpus of 305 MIMIC-IV-Note discharge summaries annotated by four annotators using a written guideline and adjudicated into a single reference version. The corpus covers seven entity types, a three-valued medication-status attribute, and five medication–attribute relation types. Pairwise inter-annotator agreement reached an exact-span F-measure of 0.896 and a character-level F-measure of 0.927. Medication mentions are split between medication-list sections (55.4%) and narrative sections (44.6%), with attribute information concentrated in the list sections (2.86 versus 0.59 attribute types per mention). A supervised entity-recognition baseline achieved a micro-averaged exact-span F-measure of 0.945 against the adjudicated reference on the held-out test set. Error analysis showed that most remaining errors involved span-boundary disagreements, particularly for the reason and duration attributes. We plan to release the corpus through PhysioNet to support reproducible development and evaluation of medication information extraction systems on MIMIC-IV-Note and to provide a curated human reference for assessing automatically generated medication labels.