Open Source
Flow Control: Steering Vision-Language-Action Models with Simple Real-Time Inputs
arXiv cs.HC0 views
arXiv:2606.10180v2 Announce Type: replace-cross Abstract: We introduce flow control of vision-language-action (VLA) models, a simple and effective way to steer VLA actions in real time through generic inputs, such as a keyboard. This method can be used out-of-the-box and does not require retraining or fine-tuning VLAs. It enables relatively crude user inputs to steer a VLA to align with user intent. Flow control writes the user input into the initial condition of the VLA's flow matching action expert. Because the flow is deterministic, integration retains information about the user input while moving it toward the action distribution learned during training, so that the generated actions reflect the user's intent (high fidelity) and stay close to the VLA's own unsteered actions (high conformity). We demonstrate that flow control has many desirable properties: (1) flow control accurately and responsively steers robot actions with user inputs, (2) it is robust to suboptimal user inputs, (3) it enables users to steer VLAs to achieve significantly higher success rates and faster task completion, and (4) fine-tuning a VLA on flow control trajectories improves the autonomous policy. Together, these results provide a simple and intuitive way for users to help steer VLA actions, increasing task performance. Project page and videos: https://jcykao.github.io/flowcontrol/.
Source:https://arxiv.org/abs/2606.10180PreviousLiving with AI Companions: Sustained AI Companionship Predicts Lower Well-Being Through Lower Human InteractionNextUnified Agentic Video Editing Across Levels of Complexity and Creativity
Related posts

Open Source Didn't Ban AI Code. 83% of Its AI Policies Allow It, With One Catch
0dev.to Git
Orbit: A High-Performance Terminal Workspace & System Monitor in Rust with WASM Plugins
0dev.to Rust

Forecasting the U.S. Senate with Prediction Markets and Monte Carlo Simulations
0dev.to AI
VoiceStudio: open-source, fully-local ElevenLabs alternative
0Hacker News New
Show HN: X Mode – macOS-style window management for Omarchy
0Hacker News Show
Show HN: Orca Style Workspaces for Herdr
0Hacker News Show