Open Source
DRIFT: Detachable Reasoning for Runtime-Flexible and Efficient End-to-End Driving
arXiv cs.RO0 views
arXiv:2611.24719v1 Announce Type: cross Abstract: The integration of vision-language models (VLMs) into end-to-end (E2E) autonomous driving has demonstrated strong potential by providing scene description and reasoning capabilities. However, VLMs are computationally expensive, and switching language reasoning on and off at runtime without hurting closed-loop performance has not been demonstrated, which hinders practical adoption. To address these issues, we propose DRIFT, an E2E autonomous driving framework that integrates a lightweight language model (LM) for visual question answering and action commentary. By efficiently projecting intermediate driving representations into the language embedding space, we demonstrate that a compact LM (<150M parameters) can achieve strong scene description and action commentary. Furthermore, we introduce a runtime-flexible planning module that is trained on surrogate reasoning features distilled from the LM and can be conditioned on either the LM or the surrogate reasoning features at inference. This design enables bypassing the LM entirely when required by deployment constraints, without sacrificing driving performance. With only 255M parameters, DRIFT reaches up to a Driving Score (DS) of 97.03 and a Success Rate of 89.39% (mean over three runs) on Bench2Drive by interleaving its LM and surrogate reasoning at a chosen reasoning rate. This is on par with the strongest non-language models while being far more compact than existing VLM-based models. DRIFT further achieves competitive results on the real-world NAVSIM and AlpaSim benchmarks at significantly lower computational cost than existing VLM-based approaches. Project page: https://drift-e2e.github.io/
Source:https://arxiv.org/abs/2611.24719PreviousALIVE: Bring Foundation Features to Life with EventsNextDoes the Action Matter? Probing Action-Conditioned Semantic Forecasting as a World Model for Autonomous Driving
Related posts
WorldComposer: Generative High-Fidelity Simulation with Digital Cousins for Generalizable Robot Learning and Evaluation
0arXiv cs.RO
Does the Action Matter? Probing Action-Conditioned Semantic Forecasting as a World Model for Autonomous Driving
0arXiv cs.RO
Show HN: Copilot Instructions Sanity Check
0Hacker News New
Show HN: A self-hosted OSS registry for Hugging Face models and datasets
0Hacker News New
What is the future of cross-border payment
0Hacker News Ask
Show HN: HL7Guard – HL7v2 regression testing for CI (validate, deidentify, diff)
0Hacker News Show