alexkstern/fineweb-nanochatbpe-20B
fineweb-nanochatbpe-20B FineWeb-Edu (from karpathy/fineweb-edu-100b-shuffle), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin train 20,000,000,000 val.bin val 52,336,096 train and val are disjoint held-out partitions. Each .bin is a raw little-endian uint16 stream (no header); token count = filesize / 2, and train.meta.json / val.meta.json carry the… See the full description on the dataset page: https://huggingface.co/datasets/alexkstern/fineweb-nanochatbpe-20B.
| Author / organization | alexkstern |
|---|---|
| Downloads | 198 (as reported by Hugging Face) |
| Likes | 0 |
| Created | 23 May 2026 |
| Last updated at source | 23 May 2026 |
Popularity, last 3 days
198 now (-1 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Recent changes
- 16 Oct 2026 Description: fineweb-nanochatbpe-20B FineWeb-Edu (from karpathy/fineweb-edu-100b-shuffle), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin → fineweb-nanochatbpe-20B FineWeb-Edu (from karpathy/fineweb-edu-100b-shuffle), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin
Changes detected by Silytics between refreshes of the source.
More from alexkstern
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.