GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Video
While multimodal Large Language Models (MLLMs) excel at offline video understanding, an interesting question of how far they are from serving as a real-time procedural coach remains unknown. Such a role typically requires an MLLM to continuously monitor the execution, detect mistakes, and provide corrective guidance in a closed-loop interaction. In this paper, we construct GuideMe, the first multi-domain benchmark for streaming video that supports training and evaluation of MLLMs for closed-loop interactive task guidance. It comprises 2,458 videos spanning 223.7 hours across diverse domains (\eg, cooking, object manipulation, daily-life guidance, and fitness), with 47,775 interaction samp...
| Authors | Fang Liu, Jinpeng Chen, Ke Xu et al. |
|---|---|
| Primary category | cs.CV |
| Published | 3 July 2026 |
| Last updated at source | 3 July 2026 |
Connected in the knowledge graph
- alisn/GuideMe_BenchmarkDataset · cited by
Models and datasets referencing this paper
- alisn/GuideMe_BenchmarkDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.