chengjunyan1/smollm-12.5-corpus
SmolLM-1/8-Corpus Around 1/8 upper-quality subset of SmolLM Corpus for training Chinchilla-optimal GPT-2 scale (sub 1.5B) models, which is a good scale for verifying a model architecture under the scaling laws. Firstly filtered samples with int_score >=4 from FineWeb-edu-dedup, then keep the training mixture with the same distribution from SmolLM. In which FineWeb-Edu-dedup occupies around 70% of the corpus. Then sample other dataset based on the mixture ratios respectively.… See the full description on the dataset page: https://huggingface.co/datasets/chengjunyan1/smollm-12.5-corpus.
| Author / organization | chengjunyan1 |
|---|---|
| Downloads | 462 (as reported by Hugging Face) |
| Likes | 7 |
| Created | 7 August 2024 |
| Last updated at source | 23 September 2024 |
Popularity, last 3 days
462 now (+7 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Recent changes
- 16 Oct 2026 Description: SmolLM-1/8-Corpus Around 1/8 upper-quality subset of SmolLM Corpus for training Chinchilla-optimal GPT-2 scale (sub 1.5B) models, which is a good scale for verifying a model architecture under the scaling laws. Firstly filtered samples with → SmolLM-1/8-Corpus Around 1/8 upper-quality subset of SmolLM Corpus for training Chinchilla-optimal GPT-2 scale (sub 1.5B) models, which is a good scale for verifying a model architecture under the scaling laws. Firstly filtered samples with
Changes detected by Silytics between refreshes of the source.
More from chengjunyan1
- chengjunyan1/smollm-10Dataset
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.