Backend
JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models
arXiv cs.CR0 views
arXiv:2609.31142v2 Announce Type: replace Abstract: Typed decision models (TDMs), such as Jev, answer a question about an input with a probability, an option, or an ordinal level. Web services execute these decisions automatically above a confidence threshold. An attacker who writes part of the input can change a decision that the service then executes. Current adversarial benchmarks cannot measure this risk, because they judge generated text. A TDM writes no text, and its wrong decisions have a valid format. We present JevAdvBench, to our knowledge the first unified benchmark for the adversarial robustness of TDMs. It contains 862 questions with audited answers and twelve black-box attack methods that edit the input or the options by adaptive search. Because a changed decision is not always wrong, a review keeps only 5,586 answer-preserving variants, and we attack only correct decisions. The attacks flip 46.3% of Jev's correct decisions, while identical re-runs flip only 0.17%. A flawed argument without a command flips 77.2%, close to direct prompt injection at 92.0%. The same variants flip 34.0% to 51.7% of correct decisions on eight open TDMs. At a 0.8 threshold, 23.5% of the decisions that the service still executes are wrong. On Agentgateway, the Jev guardrail rejects a jailbreak request but accepts it with an inserted argument. Web services should therefore treat TDM inputs as untrusted and should not act on confidence alone. Project website: https://JevAdvBench.github.io/JevAdvBench/
Source:https://arxiv.org/abs/2609.31142PreviousA Corpus of Real Scam- and Spam-Call Conversations from an Active Voice-Agent HoneypotNextA Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Services
Related posts
Laravel API Security Checklist 2026 for Mobile Apps
0dev.to

Part 2: From Five Stars to One Sentence: Semantics for Custom Compose Controls
0dev.to
A Four-Line Fix for a 9.8: Anatomy of Linux's TCP Handshake Use-After-Free
0dev.to
The Building Is a System: What Software Engineers Can Learn from Construction Site Failures
0dev.to

Part 4: When the Screen Changes Silently: Live Regions, Headings and Focus in Compose
0dev.to
FlatCake: Why Tiered KV Cache Storage Is Holding LLM Serving Back
0dev.to