AI & ML
Optimal Communication for Distributed Sparse Histogram Aggregation
arXiv math.IT0 views
arXiv:2611.04285v1 Announce Type: new Abstract: Histogram estimation is a fundamental primitive in distributed data analysis, with applications in counting queries and federated learning. In many settings, the universe of categories is enormous, yet each user's data involves only a small subset, yielding sparse local histograms. This raises a natural question: To what extent can local sparsity reduce the communication needed to estimate the global histogram? We answer this question by characterizing the communication complexity of distributed sparse histogram estimation under prescribed $\ell_1$ distortion and failure probability constraints. Our upper bound is achieved by a randomized covering scheme that optimizes how accurately each user represents its local histogram to reduce communication. By exploiting the distortion budget directly rather than imposing unbiasedness, the scheme achieves lower communication costs than standard unbiased sampling and hashing constructions. We establish the scheme's first-order optimality through a lower bound based on local support identification. The key step is to relate communication cost directly to the number of active coordinates revealed by each user's message, rather than to bound information about the aggregate via distance-based Fano's inequalities. This characterization further quantifies how the communication advantage of heterogeneous over homogeneous encoders varies across parameter regimes.
Source:https://arxiv.org/abs/2611.04285PreviousHardware-Efficient Reinforcement-Learned Scheduling for Quantum LDPC DecodersNextPairwise Distortion Distribution with Graph-Based Importance Sampling for Quantization Error and Intrinsic Dimension Estimation
Related posts
The Sandbox Fights Back: Joe Security Launches Adaptive Detonation
0HackerNoon
We Stopped Feeding LLMs Raw HTML: Cutting Agent Scraping Costs by 74% Across 10,000 Targets
0HackerNoon

What Happens When the Model Reads It Wrong: Building LLM Compliance Review That Survives an Audit
0HackerNoon
The Blue Wave Has Tech Bros Scrambling for Data Centers in Space
0Wired
Hackers exploited iOS and Android flaws to infect phones
0Ars Technica

Warmup Did Nothing for My Learning Rate — Then It Fully Rescued the Wrong One
0dev.to