OmniStepBench: Deployment-Aware Agent Evaluation across Modalities and Task Complexity
As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to assess multimodal perception, multi-step execution, tool use, and artifact delivery. However, existing benchmarks are often tied to specific task types, execution environments, or scoring protocols, limiting their comparability, interpretability, and reliability for deployment decisions. We introduce \textbf{OmniStepBench}, a benchmark for deployment-aware agent evaluation across modalities and task complexity. Built on a shared OpenClaw execution environment, OmniStepBench organizes 233 tasks selected and adapted from 22 source benchmarks i...
| Authors | Yu Liu, Zhilin Liu, Zhiwei Yang et al. |
|---|---|
| Primary category | cs.AI |
| Published | 5 September 2026 |
| Last updated at source | 14 October 2026 |
Connected in the knowledge graph
- SeerRay-Lab/OmniStepBenchDataset · cited by
Models and datasets referencing this paper
- SeerRay-Lab/OmniStepBenchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.