Dataset · Hugging Face

aakkaasshh/vaigai-dataset

Name: vaigai-dataset
Creator: aakkaasshh
License: cc-by-sa-4.0
Keywords: task_categories:text-generation, task_categories:fill-mask, multilinguality:multilingual, source_datasets:wikimedia/wikipedia, source_datasets:allenai/c4, source_datasets:MBZUAI/Bactrian-X, language:bn, language:en, language:gu, language:hi, language:kn, language:ml, language:mr, language:or, language:pa

Vaigai Dataset aakkaasshh/vaigai-dataset is an Indic-focused multilingual text corpus aggregated for tokenizer training, built with the goal of reaching tokenizer quality on par with dedicated Indic tokenizers (e.g. Sarvam AI's). One Parquet file per language is stored at the root of this repo (e.g. bn.parquet), with no nested folders. Every time new data is fetched for a language, it is merged with that language's existing file and deduplicated on the text column, so re-running… See the full description on the dataset page: https://huggingface.co/datasets/aakkaasshh/vaigai-dataset.

View on Hugging FaceReport an issue

Author / organization	aakkaasshh
License (as declared)	cc-by-sa-4.0
Downloads	307 (as reported by Hugging Face)
Likes	0
Created	1 July 2026
Last updated at source	18 July 2026

tokenizerindic-languagesmultilingual

Embed this entry

<iframe src="https://silytics.si/embed/item/a2cc3f34-1871-4af6-a96f-d5907f7a7c0f" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Popularity, last 3 days

307 now (+9 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.

More from aakkaasshh

aakkaasshh/mysdxl-datasetDataset

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.