alexkstern/c4-nanochatbpe-10B
c4-nanochatbpe-10B C4 (en) (from allenai/c4), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin train 10,000,000,000 val.bin val 168,272,017 train and val are disjoint held-out partitions. Each .bin is a raw little-endian uint16 stream (no header); token count = filesize / 2, and train.meta.json / val.meta.json carry the full metadata. The tokenizer/… See the full description on the dataset page: https://huggingface.co/datasets/alexkstern/c4-nanochatbpe-10B.
| Author / organization | alexkstern |
|---|---|
| Downloads | 281 (as reported by Hugging Face) |
| Likes | 0 |
| Created | 23 May 2026 |
| Last updated at source | 23 May 2026 |
Popularity, last 3 days
281 now (+7 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Recent changes
- 16 Oct 2026 Description: c4-nanochatbpe-10B C4 (en) (from allenai/c4), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin train 10,000,000,000 val.bin val → c4-nanochatbpe-10B C4 (en) (from allenai/c4), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin train 10,000,000,000 val.bin val
Changes detected by Silytics between refreshes of the source.
More from alexkstern
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.