Dataset · Hugging Face

izlley/llm0to1-pt-aihub-624

Name: llm0to1-pt-aihub-624
Creator: izlley
License: other
Keywords: language:ko, license:other, size_categories:10K<n<100K, format:json, modality:text, library:datasets, library:dask, library:polars, library:mlcroissant, region:us, llm0to1, pretrain, korean

AI-Hub 웹데이터 기반 한국어 말뭉치 (전처리) LLM0to1-10b (SmolLM3 기반 10B 한/영 이중언어 LLM) 사전학습 코퍼스의 일부. huggingface.co/izlley 개요 카테고리: korean 원본 출처: AI-Hub 데이터셋 624 (웹데이터 기반 한국어 말뭉치) 라이선스: AI-Hub 이용약관(재배포 허가 확인) 토큰 수(우리 토크나이저 vocab 160k): 3.777B / 문서 120,000건 600B 믹스 내 역할: korean 카테고리(목표 25% = 150B). 카테고리 unique 62.5B 중 이 소스 6.0%(~9.07B 기여), 카테고리 전체 약 2.40 epoch 반복 전처리·필터링 zip 스트리밍 추출→한글비율≥0.25·길이≥40 필터→문서 exact-dedup(md5)→PII 스크럽(주민번호·전화·이메일) 토크나이저:… See the full description on the dataset page: https://huggingface.co/datasets/izlley/llm0to1-pt-aihub-624.

View on Hugging FaceReport an issue

Author / organization	izlley
License (as declared)	other
Downloads	380 (as reported by Hugging Face)
Likes	0
Created	15 July 2026
Last updated at source	15 July 2026

llm0to1pretrainkorean

Embed this entry

<iframe src="https://silytics.si/embed/item/0b432c4b-0774-47a0-86d2-dd1eeb6db063" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Popularity, last 3 days

380 now (-31 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.

More from izlley

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.