uctnlp/mzansi-text-tokenized
MzansiText Tokenized Document-tokenized MzansiText used by the later SALLM pretraining runs. Tokenizer: custom 65,536-vocabulary BPE tokenizer Splits: 3,943,584 train, 19,379 validation, and 19,341 test rows Each raw document remains one row and is truncated to at most 2,048 tokens Sequence lengths are variable; this is not the fixed-length packed/chunked artifact used by the original MzansiLM-125M run The full corpus includes WURA, ParaCrawl, and the isiXhosa/isiZulu mC4… See the full description on the dataset page: https://huggingface.co/datasets/uctnlp/mzansi-text-tokenized.
| Author / organization | uctnlp |
|---|---|
| License (as declared) | apache-2.0 |
| Downloads | 426 (as reported by Hugging Face) |
| Likes | 0 |
| Created | 11 January 2026 |
| Last updated at source | 10 August 2026 |
Popularity, last 3 days
426 now (-3 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Connected in the knowledge graph
Related papers
More from uctnlp
- uctnlp/mzansi-textDataset
- uctnlp/mzansi-text-deduplicatedDataset
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.