Frontend
ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies
arXiv cs.RO0 views
arXiv:2610.04607v2 Announce Type: replace Abstract: Robots need to anticipate how their actions will change the world, since manipulation success hinges on the resulting contacts and object motions. However, existing Vision-Language-Action (VLA) policies that predict future observations from shared features leave the forecast decoupled from the actions the policy will actually execute, and impose no physical constraints on how the scene may evolve. We introduce ForeAct3D, a framework for policy-grounded future world modeling within VLA policies. Learnable geometric queries decode depth, semantic segmentation, and camera pose from the policy representation into current and future semantic 3D scene states, and the future queries are conditioned on the policy-generated action chunk to ground the forecast in the planned interaction. A physical-consistency closure relates the two states through background staticity and instance-level rigidity, and anchors the wrist-camera pose to end-effector kinematics. These objectives shape the shared representation used for action generation during training, and no future prediction is required at inference. Without robot pretraining, ForeAct3D achieves 98.3% average success on LIBERO and an average task length of 3.73 on CALVIN, outperforming its base policy on every suite. Ablations show that semantic 3D supervision, physical consistency, and action conditioning each improve manipulation performance, and that action conditioning substantially improves future object localization. Real-world experiments on spatial placement, object insertion, and sequential manipulation further raise average success from 6.7% to 37.8% over the base policy. The project page and code are available at https://github.com/anthonytao80-crypto/ForeAct3D.
Source:https://arxiv.org/abs/2610.04607PreviousGOTT: Object-centric Dexterous Manipulation with a Reusable Cross-Embodiment PrimitiveNextTalk, Render, Act: Integrating Social Gesture and Digital Face with Synchronized Speech for Conversational Humanoid Robot
Related posts
The User Cannot See Which Tab Is Selected, and Their Operating System Deleted Your Gradient
0dev.to CSS
Why I Prefer Lightweight WordPress Development and Custom Code
0dev.to CSS
Constraining agent CSS to design tokens (without pasting the whole theme)
0dev.to CSS
Exporting a Solo site: fixing fonts, image paths and forms by hand
0dev.to CSS
Your AI Coding Agent Needs More Than a Good Prompt
0Hacker News New
How to Migrate WordPress to Next.js Without Losing SEO
0dev.to