orionweller/mmBERT-pretrain-p2-fineweb2-remaining

mmBERT Pre-training Data P2 [!CAUTION] 🚨 DEPRECATED: KNOWN BUG 🚨 This is an old version of the data with a critical bug: token IDs were stored as uint16 and overflow. mmBERT's vocabulary is larger than 65,535, so any token ID above that limit wraps around and is silently corrupted. ❌ Do NOT use this data for training. This card is kept for reference only (data mixture, sources, and documentation). ✅ Fixed data: orionweller/mmBERT-pretraining-data-chunk0 ℹ️… See the full description on the dataset page: https://huggingface.co/datasets/orionweller/mmBERT-pretrain-p2-fineweb2-remaining.

<iframe src="https://silytics.si/embed/item/2520660f-460b-4446-af44-628126bd2a15" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 14 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.