Frontend
VLM-Guided Contact Retargeting and Generative Partner Modeling for Humanoid-Human Interaction
arXiv cs.RO0 views
arXiv:2610.28523v1 Announce Type: new Abstract: Humanoid-human interaction requires robots to operate under severe partial observability: during close physical contact, the human partner is occluded, truncated, or outside the robot's field of view, causing pose estimation to fail and often necessitating external tracking or privileged human state. At the same time, heavy mutual occlusions introduce errors into the human-human motion-capture data used for learning. We present REACT, a framework for learning deployable humanoid-human interaction policies from human-human demonstrations. REACT treats semantic contact structure as the key supervisory signal connecting demonstration processing, interaction learning, and onboard control. First, a VLM-guided contact modeling and retargeting pipeline identifies semantically meaningful contacts and transfers them onto humanoid-human geometry while preserving interaction structure. Using these annotations, privileged interaction policies learn physically grounded behaviors through interaction-aware contact rewards and are then distilled into deployable egocentric depth policies trained entirely from onboard sensing using photorealistic human depth rendering. To address severe partial observability, we introduce generative partner-state modeling objectives that predict partner motion from occluded egocentric views. We evaluate REACT in simulation and on a real Unitree G1 humanoid across five contact-rich interactions. Semantic contact modeling substantially improves preservation of demonstrated interactions, while generative partner-state modeling improves contact recall on four of five interactions, most under heavy occlusion. On real hardware, REACT performs all five interactions using onboard sensing alone.
Source:https://arxiv.org/abs/2610.28523PreviousSuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World ModelNextWhen Does Training on Synthetic Data Provably Work?
Related posts
Trailing slashes in a static Astro site on Vercel: five layers and a check that never failed
0dev.to
Try four admin templates before you install one
0dev.to
I scanned the GitHub Actions setup of 139 popular JS/TS repos. Only 1 passed 3 basic checks.
0dev.to GitHub
Power BI for Beginners: 5 Essential Concepts You Should Know
0dev.to Data
Per-Author Comment Macros for Co-Authors in LaTeX
0dev.to Productivity
How Singers Use iPhone Notes to Track Vocal Development and Performance Insights
0dev.to Productivity