Video2Skill: From Streaming Experience to Reusable Embodied Skills
Manipulation behaviors vary widely across objects and scenes, but they share a small set of reusable skills, and planning with these skills helps embodied agents generalize to new tasks. Yet an agent can only plan with skills it knows. Recovering skills from observed experience, the inverse of planning, builds this knowledge over time and yields skill data for training future agents. Vision-Language Models (VLMs) describe individual manipulation events well, but can they organize a stream of events into reusable skills? We formulate this problem as Streaming Embodied Skill Discovery (SESD): a model watches videos in sequence and maintains a persistent skill library that shapes its later d...
| Authors | Jianshu Zhang, Ce Zhang, Xiyuan Yang et al. |
|---|---|
| Primary category | cs.CL |
| Published | 29 September 2026 |
| Last updated at source | 29 September 2026 |
Connected in the knowledge graph
- Sterzhang/video2skill-benchDataset · cited by
Models and datasets referencing this paper
- Sterzhang/video2skill-benchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.