Benchmarking Visual State Tracking in Multimodal Video Understanding
Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacity for visual state tracking is fundamental to video understanding, yet remains underexplored in current evaluations of Multimodal Large Language Models (MLLMs). We introduce Visual STAte Tracking benchmark (VSTAT), a video-based benchmark designed to diagnose visual state tracking in MLLMs. VSTAT consists of 834 clips drawn from both synthetic and real-world videos, paired with 1,500 questions that cannot be answered from any single frame or short segment, requiring continuous perception and integration of events across the entire video st...
| Authors | Sihyun Yu, Nanye Ma, Pinzhi Huang et al. |
|---|---|
| Primary category | cs.CV |
| Published | 2 June 2026 |
| Last updated at source | 2 June 2026 |
Connected in the knowledge graph
- nyu-visionx/vstatDataset · cited by
Models and datasets referencing this paper
- nyu-visionx/vstatDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.