Beyond Multimodal Alignment: Shared Physical Representations Across Sensors and Action Orders
Multimodal world models are often evaluated by whether different sensors produce similar representations. However, similar representations do not necessarily imply that the models make the same physical predictions, or that those representations can be reused when actions are combined in a new order. We study both questions through the physical responses predicted by a model. We first use the Cluster Haptic dataset to ask whether audio and acceleration can independently recover the behavior of the same surface from different observations. Predictions from the two sensors are substantially closer for the same surface than for different surfaces, with a $4.5\times$ gap on average, while bot...
| Authors | Kaizhen Tan, Xin Xu, Siru Tao et al. |
|---|---|
| Primary category | cs.LG |
| Published | 19 August 2026 |
| Last updated at source | 27 September 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.