Security
AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation
arXiv cs.CR0 views
arXiv:2608.25667v2 Announce Type: replace Abstract: The integration of Large Language Models (LLMs) into cybersecurity has transformed vulnerability assessment, but it has also produced a trustworthiness crisis driven by the unchecked proliferation of "AI slop." These artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports, impose a cognitive burden on human triage pipelines that mirrors a denial-of-service attack. This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage. We formalize a taxonomy of AI slop grounded in a structured literature review and dissect its root cause: the gap between the causal deductive reasoning of security experts and the autoregressive probabilistic generation of current LLMs. We operationalize this gap through a measurable proxy, the Deductive Coverage Score, and show that chain-of-thought prompting and tool-using agents narrow but do not close it. We review mitigation strategies and argue that passive detection and watermarking target provenance rather than correctness, facing fundamental entropy constraints. We instead advocate for active neuro-symbolic verification, mapping each pipeline component to prior systems with documented limits on security inputs. Finally, we specify two evaluation instruments, CVE-Bench and Slop-Score, including dataset construction, metric formulas, and anti-gaming provisions. By shifting evaluation from linguistic fluency to mathematical verifiability, this survey provides a roadmap for securing emerging AI-driven triage systems.
Source:https://arxiv.org/abs/2608.25667PreviousHackerSignal: A Multi-Source, CVE-Linked Corpus and Prospective Benchmark for Cyber Threat IntelligenceNextA Cyber Range Evaluation of Inductive Autonomous Network Incident Response Agents
Related posts

Crypto Swaps Explained: How to Exchange USDT, TRX, and Other Tokens Without Making Costly Mistakes
0dev.to
HackerSignal: A Multi-Source, CVE-Linked Corpus and Prospective Benchmark for Cyber Threat Intelligence
0arXiv cs.CR
Cryptanalysis of Isogeny-Based Quantum Money with Rational Points
0arXiv cs.CR
faulTPM: Exposing AMD fTPMs' Deepest Secrets
0arXiv cs.CR
SoK: Post-Quantum Cryptography (PQC) Integration in Software Development: Approaches, Gaps, and Future Directions
0arXiv cs.CR

Twenty Minutes, Ten Chains, $6.3 Billion: Anatomy of the October 2026 Binance Hot-Wallet Heist, and What Repayment Really Means
0dev.to Security