Backend
Disk-Resident Graph ANN Search: An Experimental Evaluation
arXiv cs.DB0 views
arXiv:2603.01779v2 Announce Type: replace Abstract: As data volumes grow while memory capacity remains limited, disk-resident graph-based approximate nearest neighbor (ANN) methods have become a practical alternative to memory-resident designs, shifting the bottleneck from computation to disk I/O. However, since their technical designs diverge widely across storage, layout, and execution paradigms, a systematic understanding of their fundamental performance trade-offs remains elusive. This paper presents a comprehensive experimental study of disk-resident graph-based ANN methods. First, we decompose such systems into five key technical components, i.e., storage strategy, disk layout, cache management, query execution, and update mechanism, and build a unified taxonomy of existing designs across these components. Second, we conduct fine-grained evaluations of representative strategies for each technical component to analyze the trade-offs in throughput, recall, and resource utilization. Third, we perform comprehensive end-to-end experiments and parameter-sensitivity analyses to evaluate overall system performance under diverse configurations. Fourth, our study reveals several non-obvious findings: (1) vector dimensionality fundamentally reshapes component effectiveness, necessitating dimension-aware design; (2) existing layout strategies exhibit surprisingly low I/O utilization (less than or equal to 15%); (3) page size critically affects feasibility and efficiency, with smaller pages preferred when layouts are carefully optimized; and (4) update strategies present clear workload-dependent trade-offs between in-place and out-of-place designs. Based on these findings, we derive practical guidelines for system design and configuration, and outline promising directions for future research.
Source:https://arxiv.org/abs/2603.01779PreviousHierarchical Decomposition of Separable Workflow-NetsNextSOLO: Certified-Recall Metric Similarity Search with Scan-Only Sampled Inverted Lists
Related posts
Building AgentMesh: Exploring a Shared Compute Infrastructure for AI Workloads
0dev.to AI

Exploratory Testing with AI: Beyond Predefined Test Cases
0dev.to AI
Goravel file uploads and downloads with Storage
0dev.to WebDev
Reorder PDF Pages: Free Online PDF Page Reordering Tool — How-To Guide
0dev.to WebDev
Firecrawl in Production: Surviving 10K Concurrent Scrapes on Cloudflare Workers
0dev.to WebDev
I described what I shipped. Then I checked, and it wasn't the file I sent.
0dev.to WebDev