MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization
Real repository issues routinely include visual evidence such as screenshots, error dialogs, rendered UI states, and logs, yet repository-level issue localization is evaluated mostly as a text-only task. Existing multimodal SE benchmarks evaluate end-to-end repair, entangling localization with patch synthesis and obscuring whether visual input helped, hurt, or was ignored. We introduce \textbf{MM-IssueLoc}, a controlled benchmark and evaluation protocol for repository-level localization with visual evidence. MM-IssueLoc contains 652 issue-PR instances across 23 languages, with annotations for 7 image categories and 4 relevance levels. It provides file-level and function-level gold labels,...
| Authors | Shaoxiong Zhan, Shi Hu, Boyu Feng et al. |
|---|---|
| Primary category | cs.SE |
| Published | 16 July 2026 |
| Last updated at source | 16 July 2026 |
Connected in the knowledge graph
- Jasaxion/MM-IssueLocBenchDataset · cited by
Models and datasets referencing this paper
- Jasaxion/MM-IssueLocBenchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.