WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses
Bug validation asks a coding agent to produce an executable witness for a reported bug. The witness combines a concrete input with a testing harness and exposes faulty behavior during execution. Such evidence makes audit findings actionable, yet benchmark evaluation is difficult when cases reuse public historical bugs and witnesses or require manual construction. We present WitnessGym, an automated framework for constructing bug-validation benchmarks through bug injection. It injects bugs into test-reached paths of real projects, rebuilds each project, and retains cases exposed by a construction-time witness. Bug specifications and execution adapters allow extension to additional bug type...
| Authors | Haomin Qi, Xiangzhe Xu, Yiming Huang et al. |
|---|---|
| Primary category | cs.SE |
| Published | 29 September 2026 |
| Last updated at source | 29 September 2026 |
Models and datasets referencing this paper
- HarminChee/WitnessGymDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.