Backend
The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detection
arXiv cs.CR0 views
arXiv:2608.26423v2 Announce Type: replace-cross Abstract: This paper proposes a framework for constructing a classifier as a safeguard layer, and for developing a complementary diagnostic that identifies which of the classifier's confident decisions can be trusted. This framework, the Latent Diagnostic Taxonomy, consists of (i) constructing a dimensionality-optimized classifier, in which the embedding dimensionality is empirically selected via cross-validated performance rather than fixed a priori, (ii) locating a relatively small set of latent support vectors (~ 29% of total training examples) representing influential prompts for identifying tokens that alter the classifier's predicted labels, and (iii) utilizing such tokens and their associated attack magnitudes for constructing a diagnostic taxonomy. This diagnostic taxonomy provides an end-to-end guideline for flagging prompts that require different treatments: rely Safely on the classifier's decision; flag Heuristic Bias and Heuristic Override cases; route Insufficient Context cases for further human/safety review. Applying the framework to a classifier trained on a public prompt injection dataset, we find that a substantial fraction of its confident decisions (~ 77%) are not robust to removing a single token, and that this brittleness separates into two distinct failure patterns: a confidence calibration failure and a genuinely exploitable shortcut. For each zone of the taxonomy, we also recommend strategies for remediating diagnosed prompts. We illustrate the framework as a series of steps, demonstrating how each step operates.
Source:https://arxiv.org/abs/2608.26423PreviousRegime-Conditional Verification: Correctness Estimation for Adapting and Monitoring Safety ClassifiersNextNear-Optimal Sample Complexity for Recursive Entropic Risk Reinforcement Learning with a Generative Model
Related posts
Turn Today’s Messy Notes Into Tomorrow’s Checklist With AI
0dev.to AI
A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise
0dev.to AI
How I ship fixes to open-source projects maintainers actually merge (with AI in the loop)
0dev.to AI
Prompt Injection Through Documents: How to Harden an LLM Extraction Pipeline
0dev.to AI
Keeping noindex, the sitemap and the image sitemap in agreement for programmatic pages
0dev.to AI
3 Things in AI You Should Know: Character.AI Files for Bankruptcy
0dev.to AI