Dataset · Hugging Face

moca-embed/dclm_20b

Name: dclm_20b
Creator: moca-embed
License: cc-by-4.0
Keywords: language:en, license:cc-by-4.0, size_categories:10M<n<100M, format:parquet, modality:text, library:datasets, library:dask, library:mlcroissant, library:polars, arxiv:2506.23115, arxiv:2406.11794, region:us, pretraining, text

DCLM used in MoCa Pre-training 🏠 Homepage | 💻 Code | 🤖 MoCa-Qwen25VL-7B | 🤖 MoCa-Qwen25VL-3B | 📚 Datasets | 📄 Paper Introduction This is a text pre-training dataset used in the modality-aware continual pre-training of MoCa models. It is adapted from DCLM and randomly downsampled to ~20B tokens. The dataset consists of text examples. text is a string containing text while images are left blank intentionally since there is no image available. Citation… See the full description on the dataset page: https://huggingface.co/datasets/moca-embed/dclm_20b.

View on Hugging FaceReport an issue

Author / organization	moca-embed
License (as declared)	cc-by-4.0
Downloads	185 (as reported by Hugging Face)
Likes	0
Created	12 June 2025
Last updated at source	1 July 2025

pretrainingtext

Embed this entry

<iframe src="https://silytics.si/embed/item/8e03cf6c-ccaf-4e38-9e21-42e753fb9d44" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

More from moca-embed

moca-embed/MAmmoTH-VL-Instruct-12MDataset
moca-embed/pixelprose_commonpoolDataset
moca-embed/MoCa-CL-PairsDataset
moca-embed/MMEB-train-fixedDataset
moca-embed/visrag_synDataset

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.