AI & ML
Revisiting Distribution Matching for State-to-Vision Policy Distillation in Laparoscopic Robotic Manipulation
arXiv cs.RO0 views
arXiv:2610.31010v1 Announce Type: new Abstract: End-to-end control policies trained via Reinforcement Learning (RL) have emerged as an alternative to conventional surgical robotic control approaches, with recent studies exploring state-based policies trained on privileged state observations. However, such approaches rely on precise physical states that are difficult to obtain reliably in surgical environments. Vision-based RL provides an alternative by learning manipulation policies directly from RGB observations, avoiding explicit state estimation and manually engineered perception pipelines while enabling policies to acquire task-relevant visual representations. Nevertheless, obtaining vision-based policies via pure RL methods for surgical manipulation remains challenging. Thus, imitation learning (IL) becomes another solution, yet it requires a large number of expert demonstrations, which are difficult to obtain in minimally invasive surgery. In this work, we propose a framework for state-to-vision policy distillation in laparoscopic robotic manipulation. Specifically, a state-based policy with privileged observations is employed as the teacher, and a vision-based policy learns from RGB observations and robot proprioceptive information, which requires less expert demonstrations than IL and introduces superior performance than RL methods. We conduct extensive experiments on various surgical manipulation tasks in the simulated environment. Experimental results show that, in the context of laparoscopic robotic manipulation, training vision-based policies with general-purpose RL methods is challenging, whereas our proposed state-to-vision policy distillation leads to more consistent and stable performance across tasks.
Source:https://arxiv.org/abs/2610.31010PreviousTRACE: Traversability-aware Reactive Navigation via VLM-driven Adaptive Constraints EstimationNextMulti-Tool Selection for Robotic Systems: Comparing Semantic Retrieval and LLM
Related posts

The Download: the agentic web arrives and the drive for safer roads
0MIT Technology Review

Stop counting tokens, start counting what ships
0MIT Technology Review
OpenAI is finally merging ChatGPT’s chat and Work tabs
0TechCrunch

10 Open-Source Robotics Repositories for 3D Perception, VLA Policies, and Sensor Calibration
0dev.to
My self-modifying system tried to wire a gate-passing module into its own protection enforcer. It was refused three times.
0dev.to
Last night an AI agent performed brain surgery on a mouse
0Hacker News