AI & ML
HEIR: Harness Egocentric Intent for Human-Robot Interactions
arXiv cs.RO0 views
arXiv:2611.17238v2 Announce Type: replace Abstract: For robots to assist, serve, and collaborate with humans, they must not only interpret human requests conveyed through language and multimodal intentional cues, but also execute long-horizon tasks that interleave navigation and manipulation. In this paper, we introduce HEIR, a resource for studying how to Harness Egocentric Intent for Human-Robot Interaction. HEIR pairs human speech, measured gaze, and egocentric video with robot observations and teleoperated navigation-manipulation trajectories, while each session preserves successive episodes, with each request fulfilled through multiple execution subtasks. The captured data, together with human-authored annotations, support the study of intent grounding, execution planning, and task completion within a unified interaction setting. To demonstrate the value of HEIR, we conduct extensive offline experiments to evaluate how models benefit from additional human-centric context across these tasks, together with real-robot experiments that assess execution task completion. HEIR thus provides a valuable resource for studying user-centric robot interaction in naturalistic household environments. Our demo and dataset are available online.
Source:https://arxiv.org/abs/2611.17238PreviousVOKE: Visual Object Kinematics Estimator for Dexterous In-Hand ManipulationNextBeyond Trajectory Similarity: Evaluating Surgical Planning by Protocol Adherence
Related posts
You Still Have Twelve Days of Holiday Left and You Feel Guilty Taking Them
0dev.to Career
Evaluate the Decision, Not the Prose: Building an Eval Harness for an LLM Backend
0dev.to Testing
I Wrote a 40-Line Script to Count Title Tags and Got the Answer Wrong Twice
0dev.to Programming

Website Launch Checklist: Những lỗi dễ bỏ sót trước khi bàn giao dự án
0dev.to Programming
Cross-Benchmark Transfer from RL on Agentic Coding Tasks
0arXiv cs.SE
Programmable Distributions of Semantics-Preserving Variants for Evaluating Code Agents
0arXiv cs.SE