SlayerLab/tokenizers

SlayerLab Tokenizers Normalized tokenizer artifacts collected from the contributor directories in slayerlabs/tokenizer, pinned to source commit 1a5cd2c2e4df2287b4c19b3dbf5051f5d460fdc1. The dataset contains one row per tokenizer: the 38 workshop submissions plus the canonical SlayerLab Polish 32k tokenizer by kacperwikiel. Use the Dataset Viewer to sort, filter, and compare tokenizers without navigating folders. Columns author: contributor's exact GitHub username… See the full description on the dataset page: https://huggingface.co/datasets/SlayerLab/tokenizers.

<iframe src="https://silytics.si/embed/item/79151798-7889-4b85-9e25-9644eda9abc5" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.