AI & ML
Sparse Video Generation Propels Real-World Beyond-the-View Vision-Language Navigation
arXiv cs.RO0 views
arXiv:2602.05827v2 Announce Type: replace-cross Abstract: Why must vision-language navigation be bound to detailed and verbose language instructions? While such details ease decision-making, they fundamentally contradict the goal for navigation in the real-world. Ideally, agents should possess the autonomy to navigate in unknown environments guided solely by simple and high-level intents. Realizing this ambition introduces a formidable challenge: Beyond-the-View Navigation (BVN), where agents must locate distant, unseen targets without dense and step-by-step guidance. Existing large language model (LLM)-based methods, though adept at following dense instructions, often suffer from short-sighted behaviors due to their reliance on short-horimzon supervision. Simply extending the supervision horizon, however, destabilizes LLM training. In this work, we identify that video generation models inherently benefit from long-horizon supervision to align with language instructions, rendering them uniquely suitable for BVN tasks. Capitalizing on this insight, we propose introducing the video generation model into this field for the first time. Yet, the prohibitive latency for generating videos spanning tens of seconds makes real-world deployment impractical. To bridge this gap, we propose SparseVideoNav, achieving sub-second trajectory inference guided by a generated sparse future spanning a 20-second horizon. This yields a remarkable 27x speed-up compared to the unoptimized counterpart. Extensive real-world zero-shot experiments demonstrate that SparseVideoNav achieves 2.5x the success rate of state-of-the-art LLM baselines on BVN tasks and marks the first realization of such capability in challenging night scenes.
Source:https://arxiv.org/abs/2602.05827PreviousParallel Branch Model Predictive Control on GPUsNextLong-Term Multi-Session 3D Reconstruction Under Substantial Appearance Change
Related posts

Building highly available real-time bidding workloads with AWS RTB Fabric
0AWS for Industries
Thống kê đối đầu Rayo Vallecano vs Valencia: Xu thế bế tắc và quy luật chia điểm áp đảo
0dev.to Data
Phân tích dữ liệu đối đầu Alaves vs Deportivo La Coruna: Ưu thế phòng ngự tuyệt đối của đại diện xứ Basque
0dev.to Data

Thống kê đối đầu Arsenal vs Newcastle: Thế trận phân định tuyệt đối và dấu ấn phòng ngự
0dev.to Data
Tương quan thông số vòng 8 V.League 1: Thể Công - Viettel chạm trán Sông Lam Nghệ An
0dev.to Data
Into the Breach: Show the Threat, Not the Answer
0dev.to Design