AI & ML
SimRetract: Zero-Shot Sim-to-Real Robotic Assistance for Retractor Take-Over in Open Surgery
arXiv cs.RO0 views
arXiv:2610.38234v2 Announce Type: replace Abstract: We demonstrate the first zero-shot sim-to-real transfer for a surgeon-robot collaboration task in open surgery: retractor take-over. A robot arm takes a retractor from the surgeon's hand and preserves retraction at handoff, using a policy trained only on simulated demonstrations, without human demonstrations or real-world fine-tuning. Imitation learning from real demonstrations is limited by collection and labeling costs, and by performance degradation when the environment changes. Our framework, SimRetract, generates simulated demonstrations designed for transfer: the policy observes masked instrument images and grasp geometry, learns to recover from off-nominal states, and acts with margin for execution drift. Deployed zero-shot, it grasps the retractor in 90% of 50 surgeon-defined phantom presentations and preserves retraction at handoff in 78%. Results on unseen retractors suggest potential for generalization to instruments with similar grasp geometry. Across ten policy configurations and 1,200 simulated and real trials, we localize the sim-to-real gap to components: in simulation, grasp rates across ablations span 80-96%, while on the real robot removing the instrument mask alone drops performance to 22% grasped and 12% held. Transfer-oriented design thus makes zero-shot sim-to-real transfer feasible for a surgical assistance task without human demonstrations. Code is available here: https://anonymous.4open.science/r/simretract-58D2.
Source:https://arxiv.org/abs/2610.38234PreviousI3L: Iterative Interactive Imitation Learning for Whole-Body Mobile ManipulationNextActuator-Aware Observer Inputs for Cascade ADRC of Quadrotors Under Rotor Lag, Measurement Delay and Sensor Noise
Related posts
How to Design a Guided Camera Flow People Actually Follow
0dev.to UX

Over 40% of Pull Requests Now Start With an Agent. Review Them Like It.
0dev.to GitHub
Karma.Domains adds three new sources: .id premium names, .ee registry auctions and .ro buy-now listings
0dev.to News

AI เปลี่ยนอะไรในชีวิตคนไทยเดือนตุลาคม 2026? 12 ข่าวที่ใช้ได้จริงและเรื่องที่ต้องระวัง
0dev.to News
Dropout Voltage Explained: Headroom, Measurement, and Design Impact
0dev.to Design
Stability and the Output Capacitor ESR in LDOs
0dev.to Design