AI & ML
Can AI Chat Replace Traditional Search? Evidence from a RAG-based Live Events Experiment
arXiv cs.IR0 views
arXiv:2610.28639v1 Announce Type: new Abstract: Conversational AI tools are being increasingly integrated into online search and browsing interfaces. To understand how such tools affect user behavior and search outcomes, we run a pre-registered, incentivized online experiment in which participants search a catalog of several thousand events scraped from a major live-events platform. We compare three interfaces: a classic, attribute-based filter interface (Filter), a GenAI chat interface (Chat), and a combined interface offering both (Hybrid). In both AI conditions, AI output is restricted via retrieval-augmented generation (RAG). Our main result is that, relative to the other two conditions, the Chat condition performs the worst. The effect is robust across AI models (gpt-4.1-mini, gpt-5.2 and gpt-5.4) and is driven mainly by users who perform open-ended exploration (as opposed to users searching for a specific artist, venue or sports team). Furthermore, there are no significant differences on any metrics between Filter and Hybrid. We also document two mechanisms. First, Chat creates an echo chamber: without a visible menu, users construct queries based on what they think is contained in the catalog. As a result, they frequently search for event types that are not available in the catalog, and they search too narrowly. Second, Chat leads to prompting fatigue: it shifts effort towards prompting, and away from inspecting and evaluating the output. As a result, users often pick one of the first displayed suggestions rather than examining the list more carefully. Taken together, our results suggest that AI is best used as a complementary tool to existing filter-based search rather than as a replacement.
Source:https://arxiv.org/abs/2610.28639PreviousTAGR: Task-Adaptive Geo-Semantic Retrieval across Heterogeneous Urban Delivery TasksNextTransaction Foundation Models for Retrieval-Augmented Generation over Financial Transaction Data: A Comparative Evaluation with Text Embeddings
Related posts
We checked three LLM-written paper explainers against the PDFs. Two had numbers the papers never report.
0dev.to Machine Learning
Should You Rent AI or Run Your Own?
0dev.to Machine Learning
Who's hiring this week: only 1 in 30 new tech jobs is junior, and 1 in 6 'AI jobs' is AI-training work (Oct 26-Nov 1, 2026)
0dev.to Career
ClarityHope – An Online Counselling Platform for Accessible Support
0dev.to Career
The 3 documents to have before you start any freelance project (free deposit clause inside)
0dev.to Career
How I Diagnose a Slow, Loud, or Overheating PC: A Practical Checklist
0dev.to Beginners