AI & ML
TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real
arXiv cs.RO0 views
arXiv:2611.17619v1 Announce Type: new Abstract: Human demonstrations are becoming the main source of data for dexterous manipulation, and real-to-sim-to-real pipelines multiply each demonstration in simulation. These pipelines rebuild the interaction from vision, which recovers how the hand moves. Which fingers touch, when, and how hard remain unobserved, because a hand pose shows where each finger stops on the surface rather than how far it is driven to close. This gap is a force gap: on held-out recordings, a hand policy that imitates the observed pose grips with 0.31x the recorded force, and a policy trained on physics replays of vision-tracked poses grips with 1.55x. TouchTwin closes the gap with measured human touch. Each selected TouchScale recording, captured with head and wrist RGB-D cameras and an 880-taxel glove, is rebuilt in MuJoCo. Coding agents write the objects as physics-tested programs, and a per-subject tactile hand replays the interaction with the recorded touch setting the contact and the closing force of each finger. In simulation, a policy trained on touch-grounded replays grips at 0.99x the recorded force and succeeds in 20/20 trials on held-out recordings and 35/36 unseen perturbations; a contact predictor trained on them misses 13% of real contact frames, against 32% for vision-based replays. Each replay also seeds perturbed demonstrations with simulated contact and force labels, rendered as realistic video in new views and scenes.
Source:https://arxiv.org/abs/2611.17619PreviousX-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid RobotNextOn the Utility of Pre-Trained Visual Representations for Policy Learning of Contact-Rich Insertion Tasks
Related posts
How to Trim Audio for an Online Course Lesson — Free, in Your Browser
0dev.to WebDev
Chemical Anchor Fasteners: A Practical Guide to Stainless Steel Chemical Anchor Studs
0dev.to WebDev
Automating Notion Desktop App Installation on Arch Linux
0dev.to Linux
existence-trace: a journal your AI reads, and has to prove it used
0dev.to Python
A 30-question regression suite for your customer-service chatbot (with a tiny Python runner)
0dev.to Python
The boring-infrastructure approach to running AI agents: cron, env files, launchd, and a markdown decision log
0dev.to DevOps