Open Source
What Matters for World Modeling in Vision-Language-Action Models?
arXiv cs.RO0 views
arXiv:2610.19825v1 Announce Type: new Abstract: Integrating world models into Vision-Language-Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynamics beyond reactive observation-to-action mapping. However, existing approaches vary simultaneously in their world representations, integration paradigms, and policy backbones, making it difficult to determine what truly matters for effective world model integration. To disentangle these factors, we categorize existing efforts into four representative paradigms, namely Implicit Future Alignment, Explicit Future Query, MoT Future Expert, and External World Predictor. We instantiate each within a unified framework and systematically evaluate these paradigms with diverse world representations and policy backbones across in-domain, out-of-domain, long-horizon, dynamic, few-shot, and real-world settings. Our study yields four main findings: 1) world representation effectiveness is paradigm-dependent, whereas 2D semantic representations offer robust, cross-paradigm gains; 2) no single paradigm is universally optimal, yet the decoupled MoT Future Expert offers competitive overall performance, particularly in OOD robustness; 3) world modeling consistently benefits policies built on standard VLMs, while decoupled designs exhibit greater compatibility with pretrained VLAs; and 4) world modeling improves performance over action-only baselines across multiple settings, with notable gains on long-horizon and dynamic tasks. Project page and video demonstrations are available at https://dylanmqs.github.io/WM-in-VLAs/.
Source:https://arxiv.org/abs/2610.19825PreviousLearning from Demonstrations meets Symbolic Control - A Supervisory Control Architecture for Formal Certification of Learned MotionNextFOMI: A Force-Feedback Manipulation Interface for Teaching Contact-Rich Manipulation
Related posts

My uninstaller refuses to delete anything it can't prove belongs to the app. Poke holes in my safety rules
0dev.to C#
让 AI 编程助手少返工:10 条命令规则合集(免费读)
0dev.to OpenSource
Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets
0arXiv stat.ML
I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulation
0arXiv cs.RO
Sequential Planning via Anchored Robotic Keypoints
0arXiv cs.RO
WildOS: Open-Vocabulary Object Search in the Wild
0arXiv cs.RO