ByteSpanTokenisers/finewebedu-20B
FineWebEDU 20B A copy of FineWebEDU-20B used for out tokenizer experiments. The subsets are as follows: bytelevel: the full dataset tokenized using our bytelevel tokenizer bytelevel-subset_1: a 100k-row subset of the bytelevel subset, used to train bytelevel models. bytelevel-subset_2: a 100k-row subset of the bytelevel subset, used to extract llm predictions. bytelevel-llm-data: a copy of bytelevel-subset_2 with lm predictions, used to train bytespan tokenizers… See the full description on the dataset page: https://huggingface.co/datasets/ByteSpanTokenisers/finewebedu-20B.
| Author / organization | ByteSpanTokenisers |
|---|---|
| Downloads | 292 (as reported by Hugging Face) |
| Likes | 0 |
| Created | 3 April 2025 |
| Last updated at source | 23 June 2025 |
Popularity, last 2 days
292 now (-20 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.