kothasuhas/opengenome2_tokenized_n12361963_ctx1024_char19_bos_eos_iid

OpenGenome2 character tokens, 1k context, IID split A bounded, multi-source subset of Arc Institute OpenGenome2, prepared for small causal language-model experiments. This is tokenized sequence data with one input_ids column, ready for next-token training. Size and loading 12,360,963 training rows + 1,000 validation rows, exactly 1024 tokens per row (12,658,650,112 tokens including BOS/EOS). Counts match the class DCLM dataset. Parquet stores token IDs as uint8;… See the full description on the dataset page: https://huggingface.co/datasets/kothasuhas/opengenome2_tokenized_n12361963_ctx1024_cha

<iframe src="https://silytics.si/embed/item/e5e35433-2ff4-4386-b945-a05014a4cf52" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 19 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.