AI & ML
Between Hi-Fi and MMI: the overlap of redundancy and synergy in partial information decomposition
arXiv math.IT0 views
arXiv:2611.08131v1 Announce Type: new Abstract: Partial information decomposition (PID) splits the information that two sources carry about a target into redundant, unique and synergistic parts, with nonnegative decompositions lying on a segment where redundancy measures are bounded by the positive part of the co-information and the smaller source-target information. We parameterize this segment by the overlap, the common excess of redundancy and synergy over their net balance, and show that for many of the named PID measures, their redundancy can be written as the largest positive part of the co-information over a set of reference distributions. This common form induces an ordering of those PIDs on the segment. With the smallest redundancy and synergy and the largest unique atoms of any nonnegative PID, the information-theoretic version of Hi-Fi, a feature-importance method, is the zero-overlap end of the segment for two sources, with the maximum overlap bound given by the minimum mutual information (MMI) PID. We introduce, by splitting the co-information into a pairwise maximum- entropy part and the third-order connected information, a PID that sits at zero overlap except when pairwise redundancy and triadic synergy coexist. For three or more sources the bivariate results hold on every pair of merged source groups, and among all nonnegative solutions of the lattice equations MMI and Hi-Fi still give the smallest and largest unique atoms but zero overlap on all pairs at once is often unattainable. Two inequalities across pairs are necessary in general and sufficient for three sources. For identity targets zero overlap is possible exactly when the sources' information diagram is nonnegative. In random three- and four-source distributions zero overlap on all pairs fails in 21-100% of the cases.
Source:https://arxiv.org/abs/2611.08131PreviousOperational Indirect Rate-Distortion Functions for Classification TasksNextDrift-Resilient Predictive Interference Management: Online Adaptation of an Extreme-Value Predictor
Related posts
Stablecoins: The Inevitable Currency for Autonomous AI Agents
0dev.to
스테이블코인: 자율 AI 에이전트의 필연적인 화폐
0dev.to
I built a free audio cutter, joiner and voice recorder that never uploads your audio
0dev.to
Stop Copying a String Into a MemoryStream. .NET 11 Has StringStream
0dev.to
Terraform Says for_each Keys Can't Be Known Until Apply. Here's What It Actually Wants
0dev.to
Metaの150億ドル調達が支える、Quest上の「臨床補完」VRアプリ開発のリアル — AmblyoPunchから見えるインフラの価値(lazy eye / 弱視 / 両眼視 / VR vision training)
0dev.to