jalenluorion/prefetchable-store-corpus-v3

Pre-tokenised language-model training corpus, v3 A fixed, byte-verified training corpus for small-scale language-model scaling experiments: 8,200,000,000 tokens of FineWeb-Edu sample-10BT in stream order, tokenised with the Llama-2 tokenizer (vocabulary 32,000, uint16, no BOS/EOS, documents concatenated), plus three held-out evaluation slices. Sequences are 2,048 tokens; seq_permutation.npy fixes the training order so that a run on the first N sequences is a byte-identical… See the full description on the dataset page: https://huggingface.co/datasets/jalenluorion/prefetchable-store-corpus-v3.

<iframe src="https://silytics.si/embed/item/0913f54d-bdd0-4f33-be48-229d8a9b2db9" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 19 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.