Dataset · Hugging Face

tasksource/prm800k_dpo

Name: prm800k_dpo
Creator: tasksource
License: mit
Keywords: task_categories:text-generation, task_categories:reinforcement-learning, language:en, license:mit, size_categories:10K<n<100K, format:parquet, format:optimized-parquet, modality:text, library:datasets, library:pandas, library:polars, library:mlcroissant, arxiv:2305.20050, region:us, dpo

PRM800K preference pairs (prompt / chosen / rejected) PRM800K (OpenAI, Let's Verify Step by Step) turned into clean preference pairs, ready for DPO / reward modeling (TRL "standard" preference format). Raw source: tasksource/PRM800K. Train/test follow the original MATH benchmark splits (see Splits below). from datasets import load_dataset ds = load_dataset("tasksource/prm800k_dpo", "solution") # or "step" Configs solution — whole-solution preference… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/prm800k_dpo.

View on Hugging FaceReport an issue

Author / organization	tasksource
License (as declared)	mit
Downloads	304 (as reported by Hugging Face)
Likes	4
Created	1 June 2023
Last updated at source	24 September 2026

dporlhfpreferencemathreward-modelprm800k

Embed this entry

<iframe src="https://silytics.si/embed/item/cea503d0-bac1-4ce2-aa98-b87ba1fd40c7" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

More from tasksource

tasksource/mmluDataset
tasksource/bigbenchDataset
tasksource/reclorDataset
tasksource/tasksource-jev-typed-decisionsDataset
tasksource/babi_nliDataset
tasksource/strategy-qaDataset

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.