Zyphra/dclm-dedup
DCLM-Deduped DCLM is a recently released high quality dataset that uses model-based quality filtering to filter a large subset of common-crawl for similarity to OpenHermes and other instruction-tuning datasets. For reference see the DCLM paper. The original authors of DCLM did not release fully deduplicated version of their dataset, claiming that full deduplication did not improve performance. The released version was partially deduplicated in shards. Nevertheless, when… See the full description on the dataset page: https://huggingface.co/datasets/Zyphra/dclm-dedup.
| Author / organization | Zyphra |
|---|---|
| License (as declared) | cc |
| Downloads | 2.3K (as reported by Hugging Face) |
| Likes | 24 |
| Created | 23 October 2024 |
| Last updated at source | 24 October 2024 |
Popularity, last 3 days
2,307 now (-83 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Recent changes
- 16 Oct 2026 Description: DCLM-Deduped DCLM is a recently released high quality dataset that uses model-based quality filtering to filter a large subset of common-crawl for similarity to OpenHermes and other instruction-tuning datasets. For reference see the DCLM pa → DCLM-Deduped DCLM is a recently released high quality dataset that uses model-based quality filtering to filter a large subset of common-crawl for similarity to OpenHermes and other instruction-tuning datasets. For reference see the DCLM pa
Changes detected by Silytics between refreshes of the source.
More from Zyphra
- Zyphra/Zyda-2Dataset
- Zyphra/ZydaDataset
- Zyphra/ztts1_evalDataset
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.