Dataset · Hugging Face

Neetree/fineweb10B-tokenized-custom

Name: fineweb10B-tokenized-custom
Creator: Neetree
License: other
Keywords: task_categories:text-generation, language:en, language:zh, language:ja, language:ko, language:it, language:es, language:de, license:other, size_categories:10B<n<100B, region:us, pretraining, tokenized, fineweb-edu, numpy

LittleTzu FineWeb-Edu Tokenized (Custom 65k Balanced) Tokenized shards of FineWeb-Edu (HuggingFaceFW/fineweb-edu, config: sample-10BT) for language model pretraining. This dataset stores a derived, tokenized representation of the original FineWeb-Edu corpus. It has been tokenized using LittleTzu's custom 65K balanced tokenizer, optimized for multi-domain training (English, multilingual text, math, and code) while maintaining a compact vocabulary footprint that fits within a… See the full description on the dataset page: https://huggingface.co/datasets/Neetree/fineweb10B-tokenized-custom.

View on Hugging FaceReport an issue

Author / organization	Neetree
License (as declared)	other
Downloads	312 (as reported by Hugging Face)
Likes	0
Created	25 June 2026
Last updated at source	28 June 2026

pretrainingtokenizedfineweb-edunumpycustom-tokenizerbpe

Embed this entry

<iframe src="https://silytics.si/embed/item/ae189fd6-17b0-46a8-9790-6a038da4fdd4" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Popularity, last 3 days

312 now (+5 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.