Dataset · Hugging Face

open-index/ccrawl-recrawl-domains

Name: ccrawl-recrawl-domains
Creator: open-index
License: odc-by
Keywords: task_categories:text-generation, task_categories:text-retrieval, task_categories:other, license:odc-by, size_categories:1M<n<10M, format:parquet, modality:tabular, modality:text, library:datasets, library:dask, library:polars, library:mlcroissant, region:us, common-crawl, web-crawl

Common Crawl Domain Recrawl Live fetches of the home page of every ranked domain in Common Crawl's web graph, rendered to Markdown as they are fetched What is it? Common Crawl's web graph ranks domains by how central they are, but it does not tell you what those domains actually serve today. This dataset walks that ranking from the top and fetches each domain's home page now, storing the response as one Parquet row with the body, the headers, the timing and the… See the full description on the dataset page: https://huggingface.co/datasets/open-index/ccrawl-recrawl-domains.

View on Hugging FaceReport an issue

Author / organization	open-index
License (as declared)	odc-by
Downloads	1K (as reported by Hugging Face)
Likes	0
Created	20 August 2026
Last updated at source	22 August 2026

common-crawlweb-crawlrecrawlhtmlmarkdowntextparquetopen-datadomain-ranks

Embed this entry

<iframe src="https://silytics.si/embed/item/a2ba3455-a000-4a47-8d0a-d0014ef8a46c" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Popularity, last 3 days

1,010 now (+15 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.

More from open-index

open-index/hacker-newsDataset
open-index/arcticDataset
open-index/open-githubDataset
open-index/open-libraryDataset
open-index/open-wikipedia-markdownDataset
open-index/vitcoDataset

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.