priyammaz/WrenPretraining100B

Wren Pretraining Mix (~116.14B tokens) Pre-tokenized, ready-to-train pretraining data for the Wren model — so you can skip the prep pipeline entirely and start training. Tokens are packed as Megatron-style indexed datasets (.bin = concatenated uint16 token ids, .idx = per-document offset table), each document prefixed with the <|begin_of_text|> BOS token. Tokenized with the wren_tokenizer (retrained from Llama-3, vocab ≤ 65536 → uint16). Layout… See the full description on the dataset page: https://huggingface.co/datasets/priyammaz/WrenPretraining100B.

<iframe src="https://silytics.si/embed/item/1a19c79a-b69e-4b0e-a5b8-a2ad71bcf0ce" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 19 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.