Open Source
Interactive World Simulation with Cosmos for Physical AI
arXiv cs.RO0 views
arXiv:2611.06627v1 Announce Type: cross Abstract: We present a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation in Physical AI. By streaming visual observations in response to actions and scene controls, the framework enables users and embodied agents to interact with evolving virtual environments. It supports both predicting the visual consequences of actions and generating observations from physical state maintained by an external simulator. We demonstrate this approach across camera navigation and robot manipulation, controllable video transfer, and multi-view autonomous vehicle (AV) simulation. These examples provide a shared foundation for developing interactive virtual environments for Physical AI training and evaluation. Our code and model checkpoints are available at https://github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3-examples. The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation.
Source:https://arxiv.org/abs/2611.06627PreviousReactWorld: Structuring Interaction Responses for Embodied Video GenerationNextReinforcement Learning Enables Scaling Generalist Robot Policy Improvement
Related posts
On solid angles and simplex shape measures
0arXiv cs.NA
Improving the performance of regularized Stokeslet simulations using Rosenbrock time integration methods
0arXiv cs.NA
Dynamic Strategy Selection
0arXiv stat.ML
DUP: Detection-guided Unlearning for Backdoor Purification in Language Models
0arXiv cs.CR
Catastrophic Overfitting is Weight Drift the Training Loss Cannot See
0arXiv cs.CR
cuRoboV2-Fast: Fast Torque-Limited Collision-Free Motion Planning
0arXiv cs.RO