Dataset · Hugging Face

INSAIT-Institute/BrokenMath

Name: BrokenMath
Creator: INSAIT-Institute
License: cc-by-nc-sa-4.0
Keywords: task_categories:question-answering, language:en, license:cc-by-nc-sa-4.0, size_categories:10K<n<100K, format:parquet, modality:text, library:datasets, library:pandas, library:mlcroissant, library:polars, arxiv:2510.04721, region:eu, math, sycophancy, llm

BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs We introduce BrokenMath, a benchmark designed to evaluate sycophancy in Large Language Models (LLMs) within the domain of natural language theorem proving. Sycophantic models tend to agree with incorrect premises in a user's prompt, a critical failure mode in reasoning. This benchmark contains challenging math problems with deliberately flawed premises to test and… See the full description on the dataset page: https://huggingface.co/datasets/INSAIT-Institute/BrokenMath.

View on Hugging FaceReport an issue

Author / organization	INSAIT-Institute
License (as declared)	cc-by-nc-sa-4.0
Downloads	163 (as reported by Hugging Face)
Likes	2
Created	30 September 2025
Last updated at source	7 October 2025

mathsycophancyllm

Embed this entry

<iframe src="https://silytics.si/embed/item/dd98c821-bb72-43bc-8f55-24aebe90620b" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

More from INSAIT-Institute

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.