Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agents
We introduce Forecast-Dojo, a replayable environment for benchmarking and training LLM forecasting agents. It combines resolved prediction-market questions with dated news, allowing agents to research an event and revisit their predictions at successive historical dates. The same tasks and tools support repeated evaluation, collection of training interactions, and feedback from recorded outcomes without waiting for new events to resolve. Forecast-Dojo contains 1,568 Polymarket events, split by time into training and evaluation periods, and 18.8M dated news articles. In an evaluation of 12 models, research tools lower Brier score for all 12. Forecasts also improve as events unfold, with th...
| Authors | Liqin Ye, Haorui Wang, Fardin Ahmed et al. |
|---|---|
| Primary category | cs.AI |
| Published | 24 September 2026 |
| Last updated at source | 24 September 2026 |
Models and datasets referencing this paper
- FinEredium1/Forecast-DojoDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.