Backend
VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware
arXiv cs.RO0 views
arXiv:2610.34717v1 Announce Type: new Abstract: Vision--language--action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models.benchmarking 18 open VLA models across released and alternative backends on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. Our study reveals three insights. First, action-generation strategy matters more than model size. It dominates 14 of 19 Jetson configurations, latency varies by $82\times$ on Jetson and $157\times$ on AMD, and parameter count explains at most 15% of latency variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running $\pi_{0.5}$ and GR00T N1.7 2.6--2.9$\times$ faster than PyTorch with \texttt{torch.compile}, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12\% to 39\% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.
Source:https://arxiv.org/abs/2610.34717PreviousResource-Performance Trade-offs in Intra-Aisle Robotic Picking: A Discrete-Event Simulation Comparison of Shuttle Configurations With and Without an Onboard Tote LiftNextSceneAgent: An Agentic Pipeline from 3D and 4D Captures to Simulation Environments
Related posts

Copilot Fusion Takes Local Models, and the Offline Boundary Is Somewhere Else
0dev.to AI

A CTO Checklist for Evaluating AI Features Before Production Release
0dev.to AI
Humanoid Robot Demo Makes the Case for Keeping the Human Form
0dev.to AI
Supabase anon key vs service role key: what each one opens
0dev.to Security
After the eMudhra CA compromise: audit your own TLS stack in 30 minutes
0dev.to Security
Exposed Building Automation Services: Bavaria and DACH
0dev.to Security