pietrolesci/pile-deduped-pythia-preshuffled

This dataset contains the fully prepared data, which has been tokenized and pre-shuffled, used to train the Pythia (deduplicated) models. You can find these models under the EleutherAI organisation, and they are also listed in my Memorisation-Profiles collection. This data is the same as the one found in EleutherAI/pile-deduped-pythia-preshuffled, but it is presented in a more manageable format. Instead of using the Megatron format used by the GPT-NeoX library, I have stored the data in a… See the full description on the dataset page: https://huggingface.co/datasets/pietrolesci/pile-deduped-py

<iframe src="https://silytics.si/embed/item/0c54e756-ee29-4324-8d41-ac6c34035325" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 20 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.