AI & ML
The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering
arXiv cs.IR0 views
arXiv:2610.26951v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references. We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence-grounded multi-document QA benchmarks- MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues, and pair them with a controlled synthetic two-hop haystack and an evidence-augmented NoLiMa spanning 10K-1M characters. Across ten proprietary frontier and three open-weight LLMs, EBG isolates two RAG limitations that long-context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries. EBG also exposes grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge. Data and code are available at https://github.com/debobanerjee/LC-EBG.
Source:https://arxiv.org/abs/2610.26951PreviousExpert-Guided Rubrics as Dense Reward Signals for Reinforcement Learning in Non-Verifiable DomainsNextMultiCard: Multiple Attribute Cardinality Estimation via Deep Learning
Related posts
Heated mirror reveals hidden 3D patterns in rising air near a wall
0Phys.org Tech
Infrared-transparent aerogel boosts performance of subambient daytime radiative cooling materials by up to 5.2°C
0Phys.org Tech
Tracing the online networks behind AI-generated nude images
0Phys.org Tech
Microsoft's AI Risk Report Misses This Key Detail
0Dark Reading
Introducing the Grid Protectors Program
0OpenAI Blog
The world has a new plan to save nature, but it still needs the money
0Ars Technica