Industry
Seeing What Matters: Benchmarking Task-Conditioned Actionable Spatial Belief for Manipulation
arXiv cs.RO0 views
arXiv:2611.09794v1 Announce Type: new Abstract: When should a robot look at the cup, and when at the portafilter? Spatial-reasoning benchmarks evaluate whether a model can describe a scene, but manipulation needs something more specific: the ability to acquire the particular evidence that makes the current command executable. A view that is informative for one task can be useless for another, even at the same workstation. We introduce Seeing What Matters, a benchmark for task-conditioned actionable spatial belief in simulated tabletop manipulation. Across three domains--coffee workstation, drawer/cabinet, and microwave/appliance--the benchmark provides 860 frozen evaluation cases; expanding across methods, budgets, and inference repeats yields more than 115,000 evaluator episodes under two protocols: closed-loop actionability and same-object functional out-of-distribution generalization. Each case pairs a set of candidate RGB views with a manipulation command, shared scripted skills, and hidden postconditions, so the benchmark directly measures whether a view-selection policy acquires the observations its command actually needs. We also provide a diagnostic reference pipeline that maintains a spatial-functional belief state and ranks views by missing evidence. At primary budget settings, this pipeline reaches 98.8% closed-loop and 98.9% functional-OOD success; ablating the functional or command-binding components collapses performance to near zero. Seven vision-language models, evaluated on the same cases, reach only 26-43%. Replacing the reference pipeline's coverage metadata with frozen RGB-derived scores preserves 98.5%/98.9% success; shuffling the visual scores collapses it to 15%/11%, confirming that the result depends on image content. These gaps suggest that spatial belief for manipulation should be evaluated not by scene-level completeness, but by whether it acquires the evidence the current command requires.
Source:https://arxiv.org/abs/2611.09794PreviousA wearable input/output interface learns personalized models of haptic perceptionNextGrasp-TAGS: Task-Aware Grasp Selection via Critic Distillation for Prehensile Loco-Manipulation
Related posts
Le vrai enjeu de l'AB 1709 n'est pas le temps d'écran, mais l'architecture de l'attention
0dev.to Design

Customizing the MacBook Pro Glowing Logo with Backglow
0dev.to Design
L'IA accélère la découverte de peptides antimicrobiens : et si nous apprenions enfin à voir les protéines en 3D ?
0dev.to Machine Learning

Inside Apple's Siri AI Lab: Schemas, Tools, and Skills
0dev.to Machine Learning
Removing ads, telemetry and AI extras from Windows 11: a practical checklist
0dev.to Tutorial
What IT certification exams actually cost in 2026: exam, retake and renewal
0dev.to Career