CuratorKIT : Data Curation and Synthetic Data Generation for LLM Post-Training
Data curation is a critical part of post-training pipelines for large language models, yet existing tools often treat ingestion, deduplication, synthetic generation, and quality filtering as separate stages. This fragmentation makes it difficult to audit pipeline decisions or understand why individual samples are rejected. CuratorKIT is an open-source Python library that covers this full lifecycle in a single configurable pipeline. The framework is composed of six source format readers and automatic schema detection, a pre-generation data hygiene layer for credentials, PII, and toxic content, eight LLM-powered generation tasks, three complementary quality gates with provenance-exact hallu...
| Authors | Soham Bhattacharjee, Karun Sharma, Vinay Kumar Sankarapu et al. |
|---|---|
| Primary category | cs.CL |
| Published | 19 June 2026 |
| Last updated at source | 19 June 2026 |
Connected in the knowledge graph
- arghya2030/commercecore-qwen3-1.7bModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.