Frontend
Do Vision-Language-Action Models Understand Instructions? A Mechanistic Interpretability Study on Language Grounding
arXiv cs.RO0 views
arXiv:2610.10178v1 Announce Type: new Abstract: Vision-Language-Action models are designed to generalise across environments and task descriptions, raising the question of whether their action generation actually depends on the language instruction, or whether they largely rely on visual cues and superficial correlations. Robustness to variance in the visual and linguistic observation space is critical for real-world deployment, yet VLAs lack explicit grounding modules and instead rely on the intrinsic language grounding capabilities of their Vision-Language model backbones. For this reason, we conduct a controlled mechanistic interpretability study on the language grounding capabilities of two state-of-the-art Vision-Language-Action models, $\pi_{0.5}$ and GR00T N1.7, by applying activation and attribution patching to the residual stream of the action generation modules. We systematically corrupt the task instruction of input samples of the LIBERO benchmark following five strategies: synonym replacement, semantic scaling, directional corruption, random object substitution, and empty string. Our experiments find that both models are comparatively insensitive to abstract rephrasing and to referencing non-existent objects, but react strongly to empty task descriptions and, especially, to directional language. During action generation, this sensitivity is concentrated in different loci for each model: mainly in the early, periodic cross-attention layers for GR00T N1.7, versus distributed across the earliest and selected later layers for $\pi_{0.5}$. For GR00T N1.7, directional perturbations drive some of the largest causal effects while leaving the internal representational geometry comparatively unchanged, a dissociation we do not observe clearly for $\pi_{0.5}$. Finally, the reliability of attribution patching is model-dependent: it closely tracks activation patching for GR00T N1.7 but not for $\pi_{0.5}$.
Source:https://arxiv.org/abs/2610.10178PreviousLifelong small-object navigation in changing object layouts: a benchmark and methodNextDesign of a Fully Actuated 4-DOF Robotic Finger With Joint-Specific Hybrid Remote Actuation
Related posts
What’s !important #20: GTA 6 Scroll Effects, Scrollspies, random()
0CSS-Tricks

Steel beams in three.js from the same table as the weight calculator
0dev.to
How to Add Schema Markup in Bricks Builder: Complete WordPress Guide (2026)
0dev.to
Every reveal.js gap closed in Kadal Deck 0.11
0dev.to

CSS Math Dance
0dev.to
Anthropic releases amended IPO prospectus with third-quarter financials
0Hacker News