Backend
Synchronized Rebuilds:The Hidden Cost of Concurrent Inserts in Learned Indexes
arXiv cs.DB0 views
arXiv:2610.33770v1 Announce Type: new Abstract: Nine years after learned indexes were proposed, their advantage on read-mostly data is well established, and a standard benchmark rates their best concurrent version as almost ready for use. Across more than 4000 runs of 13 indexes, we find that each design pays for concurrent inserts in a different way. Gapped-array indexes rebuild their nodes in synchronized bursts, at positions that follow from three constants in the source. Model-placement indexes use three to four times the memory, buffered indexes take two to three times as long. Three of five concurrent learned indexes lost keys or stopped making progress. Standard benchmarks miss both the bursts and the lost keys. The bursts cost far more than in B-trees because each rebuild copies a node a thousand times larger than a page and, under concurrency, blocks every operation that reaches the node. The typical B-tree fix of randomizing node density only spreads the bursts out,while smaller nodes trade them for longer run times at scale. Two standard systems techniques remove them: a buffer that exists only while a node is rebuilt, and background rebuilds.We show that on 400M real keys, these techniques reduce the number of inserts whose latency exceeds 100$\mu$s by two orders of magnitude at one thread, and by one at sixteen when spare cores run the background threads, at a third to half the run time of the buffered designs.
Source:https://arxiv.org/abs/2610.33770PreviousTranslating RDF 1.2 Graphs to Property GraphsNextGeo-Filtered Approximate Nearest-Neighbor Search on a Single Proximity Graph
Related posts
DeepSeek's Open-Source "Super-Agent" Codes Without Being Asked
0dev.to GitHub
All-in-one platform or a stack you assemble? Where do coaches and educators land?
0dev.to Discuss
I Wanted to Live on Coruscant. PAT Scared Me. Now My Alexa Lives on Mute.
0dev.to Discuss
Detect What You Can Name: PaliGemma 2 Object Detection with Transformers
0dev.to Machine Learning
When Should an LLM Router Stop Being Sticky? llm-d 1.0 Cache-Affinity Routing, Break-Even Math, and a Simulation
0dev.to Machine Learning

Understanding HackerTube: Karpathy’s Two-Hour Experiment in Better Recommendations
0dev.to Machine Learning