AI & ML
Beyond Trajectory Similarity: Evaluating Surgical Planning by Protocol Adherence
arXiv cs.RO0 views
arXiv:2601.10455v2 Announce Type: replace-cross Abstract: Surgical planning is commonly evaluated by trajectory similarity to a single observed execution, a paradigm borrowed from activity anticipation benchmarks such as Ego4D. This conflates two distinct concepts: task completion and protocol adherence and assumes similarity to a completed surgical trajectory implies a valid plan. In surgery, this assumption is flawed: the clinically relevant target is protocol adherence rather than mere completion, and the reference trajectory is only one observed execution that may itself deviate from the protocol. We therefore introduce an evaluation framework grounded in expert-defined protocol rules that directly measures protocol adherence. Under this framework, similarity-based metrics systematically fail: they penalize protocol-adherent plans that deviate from the reference trajectory, while rewarding protocol-violating plans that remain superficially similar to it. Evaluating Video-LLMs further reveals two separable failure modes: unreliable video-to-state grounding and under-constrained planning. Neither is resolved by model scaling alone, whereas structural knowledge injection provides consistent gains. Our results motivate a shift from trajectory similarity toward protocol adherence as the evaluation standard for surgical planning.
Source:https://arxiv.org/abs/2601.10455PreviousHEIR: Harness Egocentric Intent for Human-Robot InteractionsNextD-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
Related posts
Software jobs are back, but not for juniors. What I'd build if I were starting in 2026.
0dev.to Discuss
How to Calculate Real Throughput in an Automated Filling Line
0dev.to Productivity
WeightBench Tests Language Models as Weight-Space Programmers
0dev.to Machine Learning
Data Cleaning for messy dates dd-mm-yyyy
0dev.to Machine Learning

How Much GPU Memory Do I Need?
0dev.to Machine Learning
The Hugging Face Ecosystem in 2026: Transformers, Datasets, Trainer, LoRA/PEFT, Gradio & Spaces (Real Code)
0dev.to Machine Learning