AI & ML
More Efficient LLM Reranking with Whole-Pool, Setwise, Long-Context Language Models
arXiv cs.IR0 views
arXiv:2606.01782v3 Announce Type: replace Abstract: LLM-based re-rankers produce rankings through repeated local comparisons (listwise, pairwise or pointwise), requiring many sequential model calls. We study how long-context LLMs can drastically reduce this computation when the entire retrieved candidate pool fits within the context window. We introduce Whole-Pool Setwise re-ranking, where each comparison ranks all the entire candidate pool, and propose DualEnd Setwise, which jointly selects the candidates predicted to be most and least relevant. By filling the ranking from both ends, DualEnd constructs a complete ranking of 100 candidates in 50 LLM comparisons. Experiments with nine open-weight LLMs on TREC DL19 and DL20 show that this requires 59.4\% fewer comparisons than previous top-oriented windowed Setwise with heapsort and 88.8\% fewer than top-oriented windowed Setwise with bubblesort, even though those baselines target only the top-10 rankings while DualEnd targets the full ranking. DualEnd's nDCG@100 is within 0.008 of the single-end whole-pool top-oriented approach, while approximately halving its token consumption and ranking time. Across six BEIR datasets, DualEnd reduces mean token consumption and ranking time by 49.4\% and 50.8\%, respectively, relative to single-end whole-pool top-oriented approach. These results demonstrate that DualEnd Setwise enables complete re-ranking with substantially fewer LLM comparisons and competitive effectiveness across several backbones. Implementation, results, and prompt templates available at https://github.com/hanglics/Whole-Pool-Setwise.
Source:https://arxiv.org/abs/2606.01782PreviousWhen Does a Second Model Help? Cross-Model Review in LLM VerificationNextDo We Still Need Gazetteers in the Era of LLMs? Chaining Retrieval with a Spatial Neuro-Symbolic Index
Related posts
Introducing the Grid Protectors Program
0OpenAI Blog
The world has a new plan to save nature, but it still needs the money
0Ars Technica

Could data center backlash flip Florida?
0The Verge

Vivo’s first Pro Max phone proves its Ultra is still in a league of its own
0The Verge

Why Peacock is making a huge bet on YouTube
0The Verge
Ninja’s first-ever microwave just replaced two of my kitchen appliances at once
0TechCrunch