Dynamic Cluster Data Sampling for Efficient and Long-Tail-Aware Vision-Language Pre-training
The computational cost of training a vision-language model (VLM) can be reduced by sampling the training data. Previous work on efficient VLM pre-training has pointed to the importance of semantic data balance, adjusting the distribution of topics in the data to improve VLM accuracy. However, existing efficient pre-training approaches may disproportionately remove rare concepts from the training corpus. As a result, long-tail concepts remain insufficiently represented in the training data and are not effectively captured during training. In this work, we introduce a dynamic cluster-based sampling approach (DynamiCS) that downsamples large clusters of data and upsamples small ones. We firs...
| Authors | Mingliang Liang, Zhuoran Liu, Arjen P. de Vries et al. |
|---|---|
| Primary category | cs.CV |
| Published | 30 April 2026 |
| Last updated at source | 6 July 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.