Backend
Effect of Temporal Resolution on Zero-Shot Semantic Understanding of Rapid Human Actions
arXiv cs.RO0 views
arXiv:2605.00496v3 Announce Type: replace-cross Abstract: Understanding human actions from visual observations is essential for human-robot interaction, particularly when semantic interpretation of unfamiliar or hard-to-annotate actions is required. For rapid or less common activities, collecting sufficient labeled data for supervised learning is challenging, making zero-shot approaches a practical alternative for semantic understanding without task-specific training. While recent advances in large-scale pre-trained models enable such zero-shot reasoning, the impact of temporal resolution, especially for rapid and fine-grained motions, remains underexplored. In this study, we investigate temporal-resolution conditions for zero-shot semantic understanding of high-speed human actions. Using kendo as a representative case of rapid and subtle motion patterns, we propose a training-free pipeline that combines a pre-trained video-language model for semantic representation with large language model-based reasoning for pairwise action comparison. Our primary experiment evaluates full-observation raw videos from 10 to 120 Hz in 10-Hz increments. Nearest class prototype (NCP) accuracy is highest at 120 Hz, and the mean over 70-120 Hz is higher than that over 10-60 Hz. The corresponding mean prototype margin changes from -5.74 to +2.52 similarity-score points between these two halves and is also highest at 120 Hz. Secondary experiments show that tracking overlays reduce mean full-observation performance but improve mean partial-observation performance. These findings provide preliminary evidence that retaining high-temporal-resolution visual information can benefit training-free semantic understanding of rapid human actions.
Source:https://arxiv.org/abs/2605.00496PreviousShow, Don't Tell: Detecting Novel Objects by Watching Human VideosNextHow Many Demonstrations Does a VLA Need? On the Sample Efficiency of VLA Finetuning
Related posts
Show, Don't Tell: Detecting Novel Objects by Watching Human Videos
0arXiv cs.RO
HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation
0arXiv cs.RO
I kept missing papers in my own field until I stopped reading everything
0dev.to AI
AI Chatbots for Customer Support: What Actually Works
0dev.to AI
11 things I learned building a tool on Claude Code hooks
0dev.to AI

Squeezing an RTX 3050: fast and exact LLM inference with SonicBoom
0dev.to AI