Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Evaluating multi-modal LLMs on historical book illustrations

AbstractHistorical book illustrations are culturally significant multimodal artifacts that pose distinctive challenges for computational analysis, including sparse visual cues, historical reproduction techniques, and material degradation. We present the first task-specific evaluation of multimodal large language models (MLLMs) on historical book illustrations, testing five contemporary models on 1,969 children’s book illustrations across a range of visual labeling exercises and caption generation tasks. Frontier models performed strongly against human consensus annotations on concrete tasks, particularly illustration and setting detection, and mirrored human-level disagreement on entity, object and action identification. All model families struggled with accurate emotional intensity estimation. GPT and Gemini produced the strongest captions, with greater than 70% of images exhibiting no factual errors. Finally, frontier models closely recovered human distributions at the corpus level for major visual categories suggesting their appropriateness for use in large-scale studies.

More from EMNLP 2026

All EMNLP 2026 papers →