StepBench: Diagnosing Real-World Procedural Understanding in AI Agents
Evaluating an agent's planning ability remains challenging for open-domain tasks, where verifying procedures through execution is often impractical. Existing evaluations typically assess only the final plan and do not distinguish procedural errors such as missing, incorrect, or misordered actions. We introduce STEPBENCH, a benchmark of 6,885 verified procedures crawled from wikiHow across 81 open-domain categories. From each procedure, we derive 5 diagnostic tasks: four with deterministic answers involving withheld, incorrect, or shuffled steps, and one requiring next-step generation. We additionally evaluate full-plan generation to examine how diagnostic performance correlates with end-t...
| Authors | Hossein A. Rahmani, Bin Wu, Chaeeun Lee et al. |
|---|---|
| Primary category | cs.AI |
| Published | 19 October 2026 |
| Last updated at source | 19 October 2026 |
Popularity, last 2 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Recent changes
- 20 Oct 2026 Name: StebBench: Diagnosing Real-World Procedural Understanding in AI Agents → StepBench: Diagnosing Real-World Procedural Understanding in AI Agents
Changes detected by Silytics between refreshes of the source.
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 21 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.