AI & ML
Prompt-Based Abstention Fails Under Misleading Context: A Controlled Study of Small Frozen RAG Models
arXiv cs.IR0 views
arXiv:2608.22228v2 Announce Type: replace Abstract: Missing and misleading evidence are not the same problem in retrieval-augmented generation (RAG), but prompt-based abstention treats them alike. Small frozen models abstain when context is absent, not when it is misleading. We introduce GRAB-RAG (Graded Abstention Benchmark for Retrieval-Augmented Generation), a paired benchmark that tests the same questions across four context conditions (supportive, degraded, missing, and misleading) in Natural Questions and HotpotQA. In the misleading condition, we edit a gold passage to support a wrong answer and place it among other retrieved passages. We test five abstention policies on three small frozen models (3.8B-8B) across two English QA benchmarks. Models abstain reliably when evidence is missing, but under explicit abstention prompting still answer 41.6% of misleading questions, with 63% of those answers echoing the planted wrong entity. Chain-of-thought provides little additional benefit. A generator-side conflict check cuts the rate to 13.3% but discards many correct answers, while a natural language inference (NLI) verifier recovers that coverage but fails when parametric memory and the misleading passage agree on the same wrong answer. Prompt-based abstention asks whether context is sufficient, not whether it is correct. The two-pass checks are diagnostic baselines, not deployable mitigations.
Source:https://arxiv.org/abs/2608.22228PreviousFederated User Behavior Modeling for Privacy-Preserving LLM RecommendationNextLEGIT: Live Eligibility Gating over Identifier Tries for Constrained Decoding of Heterogeneous Pages
Related posts
Kusner's conjecture on equilateral sets is false
0arXiv cs.NA
Nearly Optimal Fixed-Confidence Best-Arm Identification with 1-Bit Feedback
0arXiv stat.ML
When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
0arXiv stat.ML
Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts
0arXiv stat.ML
Exact Constrained Decoding for Sequence Models: Controller-Augmented Hidden Markov Models and Beyond
0arXiv stat.ML
Kernel-based potential mean-field games with unbiased random Fourier $U$-statistics
0arXiv stat.ML