VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?
Constructing an interactive 3D open world from a user query is important. However, existing methods are primarily evaluated on idealized, simple queries, making it difficult to systematically analyze and compare how multimodal agents understand user intent, use 3D tools, and reason over textual and visual 3D world information. To this end, we propose VibeWorlding, a unified framework for benchmarking and training vibe worlding agents: a multimodal agent that can autonomously infer user intent, plan scene layout, invoke 3D tools, and reflect on the multimodal feedback in a multi-turn agent-environment interaction process. To achieve this, we first build VWE-BENCH, a benchmark of 2,616 high...
| Authors | Yansong Ning, Jingwen Ye, Zhongkai Wu et al. |
|---|---|
| Primary category | cs.AI |
| Published | 15 August 2026 |
| Last updated at source | 19 August 2026 |
Connected in the knowledge graph
- usail-hkust/VWE-BenchDataset · cited by
Models and datasets referencing this paper
- usail-hkust/VWE-BenchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.