Skip to content
conftrace_
2026 EMNLP EMNLP 2026

LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting

AbstractLLM-based forecasting systems have improved on real-world tasks like financial markets and sports outcomes, largely by strengthening search, but the final prediction step remains crude: systems typically ask an LLM to read all collected evidence and emit one answer. We call this approach Monolithic Prediction, which fits forecasting poorly: probabilities over competing outcomes carry information valuable for downstream decisions, but a holistic answer collapses the supporting evidence, discarding this information. To address the above issue, we propose LEAP Likelihood Elicitation and Aggregation for Probabilistic forecasting) to keep the LLM out of the final prediction. LEAP treats evidence items as local signals, uses likelihood parameters to estimate how each supports candidate outcomes, and aggregates them through a deterministic probabilistic model into a posterior over candidate outcomes, thereby preserving per-item contributions, yielding comprehensive forecasts, and simplifying audits. We build a benchmark covering real-world prediction, information-seeking, and browsing tasks, evaluating LEAP on our custom agent loop and practical agent CLI frameworks. Given the same evidence, LEAP outperforms Monolithic Prediction in most settings and metrics across models.

More from EMNLP 2026

All EMNLP 2026 papers →