AI & ML
TAVIS: A Benchmark for Egocentric Active Vision and Anticipatory Gaze in Imitation Learning
arXiv cs.RO0 views
arXiv:2605.07943v2 Announce Type: replace Abstract: Active vision -- where a policy controls its own gaze during manipulation -- has emerged as a key capability for imitation learning, with multiple independent systems demonstrating its benefits in the past year. Yet there is no shared benchmark to compare approaches or quantify what active vision contributes, on which task types, and under what conditions. We introduce TAVIS, evaluation infrastructure for active-vision imitation learning, with two complementary task suites -- TAVIS-Head (5 tasks, global search via pan/tilt necks) and TAVIS-Hands (3 tasks, local occlusion via wrist cameras) -- on two humanoid torso embodiments (GR1T2, Reachy2), built on IsaacLab. TAVIS provides three evaluation primitives: a paired headcam-vs-fixedcam protocol on identical demonstrations; GALT (Gaze-Action Lead Time), a novel metric grounded in cognitive science and HRI that quantifies anticipatory gaze in learned policies; and procedural ID/OOD splits. Baseline experiments with Diffusion Policy and $\pi_0$ reveal that (i) active vision generally helps, consistently across training runs, but benefits are task-conditional rather than uniform; (ii) multi-task policies degrade sharply under controlled distribution shifts on both suites; and (iii) imitation alone yields anticipatory gaze that lands on the task-relevant object, with lead times comparable to those of the human demonstrations, although head motion is less smooth than in the demonstrations, an effect of action chunking that success rate does not reveal. Code and evaluation scripts are released at https://github.com/spiglerg/tavis; demonstrations (LeRobot v3.0; ~2200 episodes) and trained baselines at https://huggingface.co/tavis-benchmark.
Source:https://arxiv.org/abs/2605.07943PreviousNovaPlan: Zero-Shot Long-Horizon Manipulation via Closed-Loop Video Language PlanningNextSafety-Critical Control for Smoothed Implicit Contact Dynamics
Related posts

Use it, or lose it 🧠
0dev.to AI

AI Daily Digest November 3, 2026: OpenAI Wires Agents into ServiceNow, Australia Books a 40,000-GPU AI Factory
0dev.to AI
How Publishers Can Test Engagement Signals to Improve On-Site Discovery
0dev.to AI
What Meta's Muse actually reads: first data from 3 tiny businesses
0dev.to WebDev
Your AI apps should answer to you, not to their lab
0dev.to Security

Warp's Breach Reached Agent Decision Logs. What One Agent Trace Actually Contains
0dev.to Security