Backend
The Price of Failure Risk: Clipped Barrier Policy Gradients with Convex Regularization
arXiv stat.ML0 views
arXiv:2411.19193v3 Announce Type: replace-cross Abstract: Many operations run on a budget that must rarely run out, such as a wear allowance. A wrapper keeps the remaining budget in the state, subtracts a barrier that grows as the budget shrinks and clips the combined reward from below at -m, and a standard policy-gradient learner trains unchanged. The clip level m is a price of failure risk, since the trained value splits exactly into utility before failure, the cost of a margin and the price times the risk, so whatever policy training returns earns the most such utility net of that cost among policies with no greater risk, up to its optimization error. Its risk exceeds a comparison policy's by at most order 1/m when charged costs stay bounded, and value or validation data certify it. The main theoretical result is exponential convergence of the population training flow under any parameter regularizer convex along the motions of the network's units and strong enough against the price, biased toward the prior policy. This extends mean-field convergence of policy gradients from relative entropy to weight decay, which covers continuous-time training of finite networks. Relative entropy adds noise that lets learning leave flat regions, weight decay pulls the parameters toward a reference, and action regularization bounds the risk change by the distance to a trusted rule. Experiments on an exact maintenance benchmark, a fuel-tank controller, a solvable flow and a benchmark-suite task measure the bounds' slack. We state when another method is better, without claiming this one outperforms them.
Source:https://arxiv.org/abs/2411.19193PreviousNuclear Norm-Regularized Bayesian Matrix CompletionNextTargeted Learning for Inference on Data Fairness
Related posts

GIMP 3.2.8 Speeds Up Preferences, Improves Text Tool
0Linuxiac
Announcing the Special Interest Group (SIG) for Ray on Accelerators
0PyTorch Blog
How NewsFlow Builds a Brief: Source Attribution, Key Points and the Queue
0dev.to News
Is your reviewer agent ever actually called? Counting it from session logs
0dev.to Productivity
StillDeep: a backend for building your own POS
0dev.to C#
Experiencing Machine Learning with Premier League Data
0dev.to Machine Learning