Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents
Modern coding agents are usually evaluated by whether they eventually produce a correct patch, but patch generation depends on an earlier context-acquisition stage: finding the repository files needed for the task. We introduce Agent Retrieval Bench, a file-level benchmark for this upstream retrieval problem. Samples are built from real coding-workflow signals and evaluated against frozen base-commit repositories, with relevance defined by what an agent needs next rather than direct query-file semantic similarity. The benchmark covers four positive-retrieval tasks: code2test, comment2context, trace2code, and edit2ripple; a fifth subset evaluates selective retrieval using natural evidence-...
| Authors | Bowen Qin, Yi Xie |
|---|---|
| Primary category | cs.IR |
| Published | 27 July 2026 |
| Last updated at source | 27 July 2026 |
Connected in the knowledge graph
- eyuansu71/agent_retrieval_benchDataset · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.