Dataset · Hugging Face

bigcode/github-commits-diff-dedup-pjjs-april

Name: github-commits-diff-dedup-pjjs-april
Creator: bigcode
Keywords: size_categories:100K<n<1M, format:parquet, modality:text, library:datasets, library:dask, library:mlcroissant, library:polars, region:us

Deduplicated Commits Deduplicated based on diff: content = '\n'.join(difflib.unified_diff( old_content.splitlines(keepends=True), new_content.splitlines(keepends=True), n=5 )) Parameters: Minimum ngram size: 5 MinHash ngram size: 5 MinHash threshold: 0.8

View on Hugging FaceReport an issue

Author / organization	bigcode
Downloads	729 (as reported by Hugging Face)
Likes	4
Created	16 April 2023
Last updated at source	17 April 2023

Embed this entry

<iframe src="https://silytics.si/embed/item/b4046212-3e34-4827-9633-0e2d541a3082" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

More from bigcode

bigcode/bigcodebenchDataset
bigcode/commitpackftDataset
bigcode/self-oss-instruct-sc2-exec-filter-50kDataset
bigcode/the-stack-smolDataset
bigcode/starcoderdataDataset
bigcode/the-stack-dedupDataset

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.