Right Click, Wrong Box: Grading GUI Grounding by What a Click Does
Leading GUI grounders score 93 to 96% on ScreenSpot-v2, and reported gaps between new and prior models are about a point. At this level the useful question is what the remaining errors are. Benchmarks count a click as correct only inside one annotated box, but interfaces often expose one action through several elements: a duplicated link, the label of a checkbox, the title next to a logo. The box is not wrong. The rule is incomplete. We call a click correct when executing it produces the same outcome as clicking the annotated target. A probe that executes and fingerprints about a thousand clicks per page turns this definition into outcome maps and an execution judge for live web applicati...
| Authors | Xiaokang Yuan, Binhui Yao, Luwei Yang et al. |
|---|---|
| Primary category | cs.CV |
| Published | 27 October 2026 |
| Last updated at source | 27 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 28 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.