CountVideoBench: A Benchmark to Evaluate Prediction Biases on the Object Counting task by Video-Language Models
Object counting in videos remains a poorly characterized capability of multimodal large language models. Counting questions are included in many existing VideoQA benchmarks, but the ground-truth count distributions in these benchmarks are heavily skewed toward small values. Across the nine benchmarks we survey, the median ground-truth count is at most five, and the three most frequent values together account for between 45% and 93% of the videos. The resulting mean accuracy is then dominated by the most frequent counts; a value-dependent bias at a rare count contributes negligibly to the average and remains invisible. We introduce CountVideoBench, a benchmark of 2,600 synthetic videos cov...
| Authors | Marc Chapus, Carlos Crispim-Junior, Véronique Eglin et al. |
|---|---|
| Primary category | cs.CV |
| Published | 20 October 2026 |
| Last updated at source | 20 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 21 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.