Hfzune/Duplicate-mix

ICMR + HITEK Full DB (Mixed) — Prebuilt Indexes + One-Click Setup Prebuilt sorted indexes for the Kzr0xx/Icmr-and-hitek dataset (2.5B rows, 11 columns, ~104 GB raw parquet). Building these indexes took ~17 hours of compute. This repo saves you that work: download + run = API live in ~1-2 hours (download speed dependent). Contents The indexes are stored as sorted parts (each < 50 GB, split at row-group boundaries, order preserved) because HuggingFace's classic HTTP… See the full description on the dataset page: https://huggingface.co/datasets/Hfzune/Duplicate-mix.

<iframe src="https://silytics.si/embed/item/b3258f4a-f731-41e0-86ca-39b90991a5ff" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 1 November 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.