Video-Based Reward Modeling for Computer-Use Agents
Computer-using agents (CUAs) are becoming increasingly capable; however, it remains difficult to scale evaluation of whether a trajectory truly fulfills a user instruction. In this work, we study reward modeling from execution video: a sequence of keyframes from an agent trajectory that is independent of the agent's internal reasoning or actions. Although video-execution modeling is method-agnostic, it presents key challenges, including highly redundant layouts and subtle, localized cues that determine success. We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets. We further propose adversarial instruction translation to synthesiz...
| Authors | Linxin Song, Jieyu Zhang, Huanxin Sheng et al. |
|---|---|
| Primary category | cs.CV |
| Published | 10 March 2026 |
| Last updated at source | 10 March 2026 |
Connected in the knowledge graph
- lime-nlp/ExeVR-53kDataset · cited by
Models and datasets referencing this paper
- lime-nlp/ExeVR-53kDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.