nikolina-p/fineweb_10BT_tokenized
Dataset card for FineWeb-Edu 10B tokenized dataset This dataset contains tokenized texts from FineWeb-Edu sample-10B HuggingFaceFW/fineweb-edu. The data was tokenized using the OpenAI's tiktoken tokenizer, and structured for efficient streaming and distributed (DDP) training. Structure The dataset follows Hugging Face’s recommended structure for efficient streaming in multi-GPU environments. It consists of two splits, where each split contains a number of shards… See the full description on the dataset page: https://huggingface.co/datasets/nikolina-p/fineweb_10BT_tokenized.
| Author / organization | nikolina-p |
|---|---|
| Downloads | 485 (as reported by Hugging Face) |
| Likes | 0 |
| Created | 30 October 2025 |
| Last updated at source | 31 October 2025 |
Popularity, last 3 days
485 now (+126 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
More from nikolina-p
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.