AI & ML
Semantic Role-Structured World Model for Physical Interaction Planning
arXiv cs.RO0 views
arXiv:2608.22294v3 Announce Type: replace Abstract: World models for robotic manipulation must capture how actions transform a scene in ways that are meaningful for task execution. Existing object-centric approaches preserve the transient identity of entities, while leaving the evolving task structure that gives those dynamics meaning largely implicit. We introduce SemRole, a world model that represents scene evolution through semantic roles: a task-conditioned structure explicitly encoding the functional status of scene entities, their task-relevant relations, and the current phase of execution. Conditioned on language, visual observations, and proprioception, SemRole uses this structured state to govern action generation and action-conditioned dynamics, predicting interaction events, relation satisfaction, and phase progression. At inference time, these predictions guide phase-aware action selection and local repair of invalid rollouts. We instantiate SemRole on both lightweight LeWM and pretrained Cosmos backbones, improving closed-loop success from 45.5% to 83.0% on LIBERO, 42.9% to 55.2% on RoboCasa, and 55.0% to 81.0% on real-robot Galbot transfer. Across backbones and domains, explicitly modeling task-semantic structure yields substantially stronger world-model-based manipulation.
Source:https://arxiv.org/abs/2608.22294PreviousFlatness Preserves Instruction Following in Vision-Language-Action ModelsNextTies at the Top: A Point-in-Time Comparison of CVSS and EPSS at a Fixed Budget
Related posts
Reading project health off a download page — faster than star count
0dev.to
Best time trackers for developers, for code and for AI
0dev.to

Serving Qwen3.5-0.8B with a 0.016% weight edit: measure the loops, pin the sampling on the server
0dev.to
Design Tokens for AI: Extract a Website's Real Colours, Fonts, Shadows and Icons So an LLM Builds On Brand
0dev.to
Trying to get our Arms around AI efficiency
0Stack Overflow Blog
Why an Embedded Game Fails on Some Phones, and How to Debug It
0dev.to