AI & ML
Expert-Guided Online Reinforcement Learning for Stable and Sample-Efficient Control of Dynamical Systems
arXiv eess.SY0 views
arXiv:2511.11131v2 Announce Type: replace Abstract: Online reinforcement learning (RL) often requires extensive interactions with the environment, which makes the training costly. Additionally, exploration can destabilise the dynamical system, which may be unsafe. On the other hand, offline learning from expert demonstrations avoids unsafe exploration but is limited by the quality and coverage of the offline dataset. In this paper, we propose a model-free RL approach that combines expert demonstrations with limited online interactions. Specifically, we optimise a policy objective consisting of an online performance term and a behavioural cloning regulariser computed from offline expert demonstrations. The regulariser encourages the learned policy to stay close to expert behaviour, reducing the need to explore unfamiliar actions. Expert demonstrations guide policy learning and reduce the need for many online interactions and exploration. For stochastic linear time-invariant (LTI) systems, we establish conditions for linear convergence of exact policy gradient updates and for all intermediate policies to remain stabilising. Although our theoretical analysis focuses on linear systems, we also evaluate the proposed approach on nonlinear systems to demonstrate its effectiveness beyond the linear setting. \rev{Numerical results show that the proposed method reaches comparable performance in fewer gradient steps and with less number of samples than a purely online RL baseline.
Source:https://arxiv.org/abs/2511.11131PreviousIntegrating Planning and Predictive Control Using the Path Feasibility GovernorNextStrong nonlinear detectability and moving horizon estimation for nonlinear systems with unknown inputs
Related posts
Doing Show HN generates a ton of automated email pitches
0Hacker News
Apple TV App to Get Video Podcasts Section Today
0MacRumors

New Apple TV 4K's A19 Chip Has One Core Switched Off
0MacRumors
How the brain decides which moments to remember – and which to forget
0Hacker News New
Microsoft – more details on the root cause
0Hacker News New
Questions swirl around FAA's new SMART AI system, though officials say no link to ground stop
0Phys.org Tech