AI & ML
Do VLAs Really Follow Instructions? Diagnosing How-Level Instruction Following in Vision-Language-Action Models
arXiv cs.RO0 views
arXiv:2610.21251v1 Announce Type: cross Abstract: Vision-language-action (VLA) models are expected to follow instructions like their language backbones, yet prior work has focused on What-level instructions that specify the task rather than How-level instructions that specify how to carry it out. Through our diagnoses, we find that How-level instructions do not help current VLAs and lower task success. We hypothesize that pairing each training episode with a single instruction limits How-level instruction following, and show that training with segment-level enriched instructions makes actions respond to such instructions without consistently raising task success. To measure How-level instruction following beyond standard benchmarks, we introduce SIMPLER-IN, where success requires conditions given only in the instruction. It shows that the gained responsiveness does not extend to demanding or out-of-distribution How-level instructions. How-level instruction following thus remains an open challenge.
Source:https://arxiv.org/abs/2610.21251PreviousRoPE-Flow: Monocular Vision-based Robot Pose Estimation via 2D Graph-Conditioned 3D Flow MatchingNextCLAD: Keypoint Matching Using Global Line Correspondences
Related posts

Build World Models for Interactive Simulation with NVIDIA Cosmos
0NVIDIA Developer Blog
Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling
0arXiv cs.RO
Shared Modular Recurrence in Contextual MDPs for Universal Morphology Control
0arXiv cs.RO
BeeTalk 群主必备:AI 群管机器人 AIGB 免费解决垃圾信息问题
0dev.to AI
I Open-Sourced a Free YouTube Shorts Factory. The Money Isn't in the Shorts.
0dev.to AI
Priced per TB/s, the H200 beats the H100 at 6 of 7 providers
0dev.to Python