UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing
As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish. Consequently, improving Large Language Models (LLMs) now depends less on data expansion and more on higher-quality data utilization. However, in the context of large-scale corpora, existing refinement methodologies face significant limitations in quality, efficiency, and reliability: Rule-based approaches are constrained by fixed heuristics and struggle with instance-level variations; LLM-based approaches improve quality but fail to meet the efficiency and reliability requirements of large-scale data processing. To address these challenges, we propose UltraX, a function-calling refine...
| Authors | Xinlong Zhao, Dongsheng Liu, Hengyu Zhao et al. |
|---|---|
| Primary category | cs.CL |
| Published | 9 July 2026 |
| Last updated at source | 9 July 2026 |
Models and datasets referencing this paper
- openbmb/UltraX-PreviewDataset
- Troonull/UltraX-PreviewDataset
- Kangarooz/UltraX-PreviewDataset
- Aquiles-ai/TinyQwen-DataDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.