Dataset · Hugging Face

jensjepsen/danish-pretokenized-16k

Name: danish-pretokenized-16k
Creator: jensjepsen
License: cc-by-4.0
Keywords: language:da, license:cc-by-4.0, size_categories:10M<n<100M, format:parquet, library:datasets, library:dask, library:polars, library:mlcroissant, region:us, pretokenized

Danish Pretrain — pretokenized (byte-level BPE, 16k vocab) — v1 partial Pretokenized version of jensjepsen/danish-pretrain via jensjepsen/danish-tokenizer. Note: This is a partial version — 94 of 96 shards. The 2 missing shards crashed during tokenization (rare pathological content). See jensjepsen/danish-pretokenized-16k-supp for the recovered rows. 93,200,464 docs Schema: input_ids: list<int32>, attention_mask: list<int8> 94 zstd-compressed parquet shards

View on Hugging FaceReport an issue

Author / organization	jensjepsen
License (as declared)	cc-by-4.0
Downloads	460 (as reported by Hugging Face)
Likes	0
Created	20 July 2026
Last updated at source	20 July 2026

pretokenized

Embed this entry

<iframe src="https://silytics.si/embed/item/3a42cb52-391e-47f1-b056-78f03ddc20b5" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Popularity, last 5 days

460 now (-95 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.

More from jensjepsen

jensjepsen/danish-tool-dialogues-v9Dataset
jensjepsen/danish-tool-worlds-v4-declaredDataset

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 20 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.