Dataset · Hugging Face

Mapika/bookiebench

Name: bookiebench
Creator: Mapika
License: cc-by-4.0
Keywords: task_categories:question-answering, task_categories:multiple-choice, task_categories:text-classification, language:en, language:de, language:es, language:pt, language:zh, language:fr, language:ru, license:cc-by-4.0, size_categories:10K<n<100K, format:parquet, modality:tabular, modality:text

BookieBench Can a model's probabilities be trusted? BookieBench asks a model for many linked probabilities about one situation. It then checks four things: coherence: whether a bookie can Dutch-book the answers, whether they are invariant to option and evidence order, and whether updates follow a martingale; calibration against sampled gold; skill against exact posteriors, relative to the evidence-blind uniform_joint and to train-fitted priors; evidence sensitivity. Code… See the full description on the dataset page: https://huggingface.co/datasets/Mapika/bookiebench.

View on Hugging FaceReport an issue

Author / organization	Mapika
License (as declared)	cc-by-4.0
Downloads	131 (as reported by Hugging Face)
Likes	0
Created	27 September 2026
Last updated at source	27 September 2026

calibrationprobabilistic-reasoningcoherencedutch-bookbayesian-inferencebenchmark

Embed this entry

<iframe src="https://silytics.si/embed/item/670b172f-2cd0-4b29-be2a-512e45ab59b0" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.