PushupBench: Your VLM is not good at counting pushups
Large vision-language models (VLMs) can recognize \textit{what} happens in video but fail to count \textit{how many} times. We introduce \textbf{PushupBench}, 446 long-form clips (avg. 36.7s) for evaluating repetition counting. The best frontier model achieves 42.1\% exact accuracy; open-source 4B models score $\sim$6\%, matching supervised baselines. We show that accuracy alone misleads -- weaker models exploit the modal count rather than reason temporally. Fine-tuning on counting with 1k samples transfers to general video understanding: MVBench (+2.15), PerceptionTest (+1.88), TVBench (+4.54), suggesting counting is a proxy for broader temporal reasoning.PushupBench incorporated in \tex...
| Authors | Shengzhi Li, Jiarun Chen, Karun Sharma et al. |
|---|---|
| Primary category | cs.CV |
| Published | 25 April 2026 |
| Last updated at source | 25 April 2026 |
Connected in the knowledge graph
- anonymousatom/pushupbenchDataset · cited by
Models and datasets referencing this paper
- anonymousatom/pushupbenchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.