AI & ML
Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power
arXiv math.IT0 views
arXiv:2611.07247v1 Announce Type: cross Abstract: Power matters for AI safety. Optimal agents tend to acquire power across a wide range of objectives (instrumental convergence). As we embed AI agents into socio-technical systems, humans risk gradual disempowerment. Both concerns call for a formal, quantitative measure of power. Two such measures in the context of Markov games are information-theoretic empowerment, the capacity of the channel from an agent's actions to future states, and capability-based individual power, a measure of goal attainment built from the probabilities that a human would attain each goal in a family of candidates were they pursuing it. We show where the two meet. From individual power, Fano's inequality yields a floor $B(s)$. This floor is a lower bound on what we term goal revelation, the mutual information $\mathbb{I}(G;\xi \mid s)$ between the human's latent goal and the realised trajectory. Empowerment caps revelation from above, so that $B(s) \le \mathbb{I}(G;\xi \mid s) \le \mathbb{I}(\mathbf{A}_h \to \xi \mid s) \le \mathfrak{E}_h(s)$. Each quantity can be large while the one below it stays at zero, highlighting the following failure modes. An assistant maximising empowerment can prefer an interface offering broad influence over goal-irrelevant degrees of freedom to one that lets the human attain every goal they might hold. An assistant maximising revelation cannot tell a working interface from one rewired so that every goal yields the wrong outcome, since relabelling goals leaves mutual information unchanged while zeroing every attainment probability. Positive guarantees therefore come only from the attainment end of the chain. A satisfaction floor, computable from two statistics of interaction logs, offers a way to monitor the power users retain in a deployed system, and simulations in random environments show the three quantities are correlated but far from interchangeable.
Source:https://arxiv.org/abs/2611.07247PreviousDithered Belief Propagation for Real-Time Quantum Memory DecodingNextGraph Neural Networks for Resource Allocation in Integrated Sensing and Communication
Related posts
HN 1위 'Subvocal — 소리 없이 컴퓨터와 대화하기'가 594 업보트를 받으며 개발자 커뮤니티를 달궜습니다. 서브보컬(아성대 근육 미세 신호)만으로...
0dev.to UX
Phân tích dữ liệu Cagliari 1-0 Frosinone: Hiệu quả tối đa của Adopo và điểm tựa Caprile
0dev.to Data

Phán Quyết Kỷ Luật Tại Giải VĐQG Morocco: Biến Động Điểm Số Của Widad Temara Và Wydad Casablanca
0dev.to Data
Switching Recovery Tools Is a Data Job Before It Is a Vendor Job
0dev.to Data
⚡ What Every Developer Forgets!
0dev.to Discuss
Moonshot's Kimi K3.1 Pro Shakes Up Western AI Coding
0dev.to Machine Learning