AI & ML
Beyond Mutual Information: Extension Profiles and Shape Functions of Random Variable Pairs
arXiv math.IT0 views
arXiv:2606.23849v2 Announce Type: replace Abstract: We study the extension profile of a pair of jointly distributed finite-valued random variables $(X,Y)$, defined as the set of all triples of numbers \[ ( H(X|W), H(Y|W), I(X:Y|W) ) \] obtained by extending the pair with an auxiliary random variable $W$. This object captures structural properties of a joint distribution that are not determined by the entropies of $X$ and $Y$ and their mutual information. We associate with the extension profile its \emph{shape function} defined in terms of the Legendre--Fenchel transform of the profile's lower boundary. We establish upper and lower bounds on the shape function in terms of classical information-theoretic quantities. For random pairs $(X,Y)$ whose joint distribution is uniform on its support, we relate the shape function to combinatorial and spectral properties of the associated bipartite support graph. We show that for random pairs uniformly distributed on the edges of balanced expander graphs, the shape function is within $O(\log H(XY))$ of the pointwise infimum over all pairs, which proves the conjecture stated in [MR26]. These results connect the extension profile with non-Shannon-type information inequalities, G\'acs--K\"orner common information, and spectral graph theory. We apply these techniques to study an approximate version of a question posed by Zhen Zhang ([Zha03]) concerning the characterization of pairs with maximally non-extractable mutual information.
Source:https://arxiv.org/abs/2606.23849PreviousNonlinear Information Theory: Characterizing Distributional Uncertainty in Communication Models with Sublinear ExpectationNextSerial Depth of Adaptive Parallel Sampling: Lower Bounds and Schedules for Masked Diffusion Models
Related posts

What Arconatese 3-4 Pontedera tells us about Serie D - Girone D
0dev.to Watercooler
Your AI agent got an HTTP error. Should it retry?
0dev.to Discuss
Is AI dangerous, or just misunderstood?
0dev.to Discuss
Why we price a developer tool per fix instead of per seat
0dev.to Productivity
My LLM-Powered Sidekick for Painless PR Descriptions
0dev.to Productivity
Reward-Adaptive Backtracking GFlowNet: A Technique Worth Understanding
0dev.to Productivity