MzansiText and MzansiLM: An Open Corpus and Decoder-Only Language Model for South African Languages
Decoder-only language models can be adapted to diverse tasks through instruction finetuning, but the extent to which this generalizes at small scale for low-resource languages remains unclear. We focus on the languages of South Africa, where we are not aware of a publicly available decoder-only model that explicitly targets all eleven official written languages, nine of which are low-resource. We introduce MzansiText, a curated multilingual pretraining corpus with a reproducible filtering pipeline, and MzansiLM, a 125M-parameter language model trained from scratch. We evaluate MzansiLM on natural language understanding and generation using three adaptation regimes: monolingual task-specif...
| Authors | Anri Lombard, Simbarashe Mawere, Temi Aina et al. |
|---|---|
| Primary category | cs.CL |
| Published | 21 March 2026 |
| Last updated at source | 21 March 2026 |
Models and datasets referencing this paper
- uctnlp/mzansilm-125mModel
- uctnlp/mzansi-textDataset
- uctnlp/mzansi-text-deduplicatedDataset
- uctnlp/mzansi-text-tokenizedDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.