AI & ML
MIPR: A Benchmark for Reasoning-Intensive Multimodal Intent-to-Product Retrieval with Multi-Category Solutions
arXiv cs.IR0 views
arXiv:2611.22509v1 Announce Type: new Abstract: Real-world e-commerce queries often express user intent without naming a target product, and a single intent may be fulfilled by products from multiple categories. However, existing multimodal retrieval benchmarks typically assume a relatively well-defined target, leaving retrieval for intents with multi-category solutions less explored. To fill this gap, we introduce MIPR, a benchmark for Multimodal Intent-to-Product Retrieval comprising 3,522 multimodal queries and a shared corpus of 30,731 products. MIPR features reasoning-intensive retrieval, multi-category solutions, and fine-grained relevance labels. Experiments show that the best embedding-based retriever achieves 49.23% nDCG@10 on the full query set, and 55.86% solution-category recall@10 on queries with multiple solution categories. Reranking improves these scores to 60.11% and 67.00%, respectively, yet substantial gaps to the candidate-pool oracle remain. These results establish MIPR as a challenging and diagnostic benchmark for evaluating multimodal intent-to-product retrieval. The data and code are publicly available at https://huggingface.co/datasets/leequeue/MIPR.
Source:https://arxiv.org/abs/2611.22509PreviousFrom Relevance to Utility: Rubric Alignment in Yuanbao SearchNextUrban Representation Learning for Identifying Potential Disparities in Crime Reporting
Related posts
The AI Token Casino: How the Regenerate Button Drains Your Budget
0dev.to AI
The Teacher in the Age of AI: Will the Machine Take Their Place, or Give Them Back Their Time?
0dev.to AI

Skillware v0.6.2: Section-Chunked Tax PDFs, RPC Health Gates & Strict Core Typing
0dev.to AI

GPT-6.2 Astra vs GPT-6 Astra: What Changes in Cost per Accepted Task?
0dev.to AI
Jobs That Fade, Jobs That Emerge: What the Labour Market Looks Like Up to 2030
0dev.to AI
Can an AI agent read your pricing page and submit your demo form? A dev checklist for agent-ready sites
0dev.to WebDev