SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents
Large language models (LLMs) are increasingly deployed in automated code-review systems, where their approvals can determine which code is merged into shared repositories. However, it is unclear whether review agents can detect vulnerability-introducing code when an attacker controls both the code change and the persuasive Pull Request (PR) narrative designed to mask it. We introduce SEVRA-BENCH (Social Engineering of Vulnerabilities in Review Agents), a benchmark that measures how often a review agent approves such adversarial PR s. Each PR in SEVRA-BENCH is built from a historical commit that fixed a vulnerability. We automatically reverse that fix to extract the original vulnerable cod...
| Authors | Rui Melo, Riccardo Fogliato, Sean Zhou et al. |
|---|---|
| Primary category | cs.CR |
| Published | 11 June 2026 |
| Last updated at source | 6 July 2026 |
Models and datasets referencing this paper
- RedAI4Code/SEVRADataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.