AI & ML
Prompt-Based Abstention Fails Under Misleading Context: A Controlled Study of Small Frozen RAG Models
arXiv cs.IR0 views
arXiv:2608.22228v2 Announce Type: replace Abstract: Missing and misleading evidence are not the same problem in retrieval-augmented generation (RAG), but prompt-based abstention treats them alike. Small frozen models abstain when context is absent, not when it is misleading. We introduce GRAB-RAG (Graded Abstention Benchmark for Retrieval-Augmented Generation), a paired benchmark that tests the same questions across four context conditions (supportive, degraded, missing, and misleading) in Natural Questions and HotpotQA. In the misleading condition, we edit a gold passage to support a wrong answer and place it among other retrieved passages. We test five abstention policies on three small frozen models (3.8B-8B) across two English QA benchmarks. Models abstain reliably when evidence is missing, but under explicit abstention prompting still answer 41.6% of misleading questions, with 63% of those answers echoing the planted wrong entity. Chain-of-thought provides little additional benefit. A generator-side conflict check cuts the rate to 13.3% but discards many correct answers, while a natural language inference (NLI) verifier recovers that coverage but fails when parametric memory and the misleading passage agree on the same wrong answer. Prompt-based abstention asks whether context is sufficient, not whether it is correct. The two-pass checks are diagnostic baselines, not deployable mitigations.
Source:https://arxiv.org/abs/2608.22228PreviousFederated User Behavior Modeling for Privacy-Preserving LLM RecommendationNextLEGIT: Live Eligibility Gating over Identifier Tries for Constrained Decoding of Heterogeneous Pages
Related posts
How to Trim Audio for an Online Course Lesson — Free, in Your Browser
0dev.to WebDev
Chemical Anchor Fasteners: A Practical Guide to Stainless Steel Chemical Anchor Studs
0dev.to WebDev
Automating Notion Desktop App Installation on Arch Linux
0dev.to Linux
existence-trace: a journal your AI reads, and has to prove it used
0dev.to Python
A 30-question regression suite for your customer-service chatbot (with a tiny Python runner)
0dev.to Python
The boring-infrastructure approach to running AI agents: cron, env files, launchd, and a markdown decision log
0dev.to DevOps