Dataset · Hugging Face

gair-prox/RedPajama-pro

Name: RedPajama-pro
Creator: gair-prox
License: apache-2.0
Keywords: task_categories:text-generation, language:en, license:apache-2.0, size_categories:10M<n<100M, format:parquet, modality:text, library:datasets, library:dask, library:mlcroissant, library:polars, arxiv:2409.17115, region:us, web, common crawl

📚 RedPajama-pro ArXiv | Models | Code RedPajama-pro is refined from RedPajama-Data-V2 using the ProX refining framework. It contains about 30B high quality tokens, ready for general language model pre-training. License RedPajama-pro is based on RedPajama-Data-V2, which is made available under an apache-2.0 license; users should also abide by the CommonCrawl ToU: https://commoncrawl.org/terms-of-use/. We do not alter the license of any of the underlying… See the full description on the dataset page: https://huggingface.co/datasets/gair-prox/RedPajama-pro.

View on Hugging FaceReport an issue

Author / organization	gair-prox
License (as declared)	apache-2.0
Downloads	194 (as reported by Hugging Face)
Likes	4
Created	16 September 2024
Last updated at source	26 September 2024

webcommon crawl

Embed this entry

<iframe src="https://silytics.si/embed/item/de5f5ea4-925f-4317-80c2-a3c7587a2b1f" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Popularity, last 3 days

194 now (-14 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.

Connected in the knowledge graph

QuantFactory/web-doc-refining-lm-GGUFModel · used to train
gair-prox/math-doc-refining-lmModel · used to train
gair-prox/web-chunk-refining-lmModel · used to train

Explore this entry in the graph

Recent changes

16 Oct 2026 Description: 📚 RedPajama-pro ArXiv | Models | Code RedPajama-pro is refined from RedPajama-Data-V2 using the ProX refining framework. It contains about 30B high quality tokens, ready for general language model pre-training. License RedPajama-pro is bas → 📚 RedPajama-pro ArXiv | Models | Code RedPajama-pro is refined from RedPajama-Data-V2 using the ProX refining framework. It contains about 30B high quality tokens, ready for general language model pre-training. License RedPajama-pro is bas

Changes detected by Silytics between refreshes of the source.

More from gair-prox

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.