Skip to content
conftrace_
2026 ICML ICML 2026

Position: Time to Close The Validation Gap in LLM Social Simulations

LLM-based social simulations—in which many language model agents are situated in social situations and interact over multiple turns—are rapidly proliferating across policy analysis, epidemiology, and computational social science. Yet the field lacks consensus on how to validate these simulations, with evaluation methods that are few, underdeveloped, fragmented, and rarely shared across disciplines. We argue this creates a serious risk: premature deployment of unvalidated simulators in high-stakes domains. Our position is that the field must pivot from expansion to consolidation, prioritizing methodological standardization—shared benchmarks, open data, and reproducible evaluation protocols grounded in social science and complex systems research. We outline a concrete research program organized around specific learning problems/benchmarks, providing a path toward answering the fundamental question: when are LLM social simulations useful modelling objects?

More from ICML 2026

All ICML 2026 papers →