Database
Distributed Datalog at Scale with Worst-Case Optimal Joins
arXiv cs.DB0 views
arXiv:2610.12692v1 Announce Type: new Abstract: Scalable logic-based analytics are frequently dominated by cyclic multi-way joins over large relational datasets. State-of-the-art systems evaluate these joins through chains of binary joins, incurring substantial overhead from large intermediate relations that must be materialized and redistributed across the network. Worst-Case Optimal Joins (WCOJ) offer a provably superior alternative by evaluating joins one variable at a time, eliminating intermediate blowup. However, existing WCOJ implementations target either shared-memory machines or one-shot, non-recursive queries on dataflow and big-data platforms, leaving WCOJ-based recursive evaluation on distributed-memory HPC systems an open problem. We present, to our knowledge, the first distributed-memory Datalog engine that uses WCOJ for recursive fixpoint evaluation. Our design builds on HyperCube partitioning, which hash-partitions relations along variable-aligned grid dimensions and replicates them along missing dimensions, and extends it from a single shuffle to iterative execution: in every semi-naive iteration, only newly derived tuples are repartitioned and re-replicated, using MPI all-to-all collectives and per-dimension all-gathers over subcube communicators, while the same layout is reused across iterations so that all WCOJ intersections remain rank-local. We evaluate our system on the Aurora and Polaris leadership-class supercomputers at Argonne National Laboratory using graph mining and program analysis benchmarks, demonstrating strong scalability, outperforming state-of-the-art Datalog engines on most benchmarks, and characterizing how partition dimensionality trades runtime against memory feasibility.
Source:https://arxiv.org/abs/2610.12692PreviousPrivacy-Preserving Intent Fulfilment and Assurance for 6G RANNextSemantic Views: Certified Reuse of LLM Labels Across Semantic Queries
Related posts
SOLID Principles Explained: 5 Essential Principles for Writing Clean, Maintainable Code
0dev.to Tutorial
How to Build a Professional Resume in 2026: Best Resume Formats, Templates, and ATS Tips
0dev.to Career
How to Explain Your Project in a Technical Interview
0dev.to Career
Stop Calling It "Refactoring" When You Are Just Moving The Bugs Around
0dev.to Career
PostgreSQL migrations that lock a busy table
0dev.to Testing

🧠 Do Prompt ao RAG: como fazer um LLM responder sem inventar (e sem processo do cliente)
0dev.to Java