AI & ML
Optimal R\'enyi and Logarithmic Additive Functional Entropy Estimation
arXiv math.IT0 views
arXiv:2610.28999v1 Announce Type: new Abstract: We characterize the minimax sample complexity of estimating R\'enyi entropy from independent samples on an alphabet of size at most $k$, when the known order $\alpha>1$ may vary with $k$. For every fixed additive accuracy $\varepsilon\in(0,1/2)$ and success probability $2/3$, we prove matching bounds with constants independent of $k$ and $\alpha$. For $1<\alpha<\log k$, the rate is $\Theta_\varepsilon(\alpha k^{1-1/\alpha}X_\alpha(k,\varepsilon))$, where a monotone scalar inequality defines $X_\alpha$ at noninteger orders. The bounds remain uniform as the order approaches one or an integer, or grows with the alphabet. Integer orders below $\log k$ have rate $\Theta_\varepsilon(\alpha k^{1-1/\alpha})$; orders at least $\log k$ and min-entropy have rate $\Theta_\varepsilon(k\log k)$. We extend the analysis to logarithms of positive additive functionals. For every fixed continuous $f$ satisfying $f(0)=0$, $f(x)>0$ for $x>0$, and $f(x)\sim ax^r$ at zero, with $a>0$ and $r>1$, estimating $-\beta^{-1}\log\sum_i f(p_i)$ at fixed accuracy, for fixed $\beta>0$, has sample complexity $\Theta(k^{1-1/r})$ for integer $r$ and $\Theta(k/\log k)$ otherwise. A replication reduction preserves the sampling experiment and transfers the rates for powers. Smooth flat functions show that regularity alone does not determine the complexity. The upper bound combines polynomial approximation, Gamma count statistics, and an independent pilot, with a global affine correction near order one. The lower bound uses interpolation to construct positive mean-one priors and controls their distinguishability and entropy separation after normalization. Exact count-variance identities and weighted approximation duality connect the two arguments.
Source:https://arxiv.org/abs/2610.28999PreviousUniversality of approximate message passing under a fourth-moment tail conditionNextEncoding Trees: A Method for Producing a Variable-Length Prefix Code From Sets of Arbitrary Size
Related posts
SaaS Partner Scout: A Script for Identifying GitHub SaaS Project Contributors
0dev.to GitHub
WARN Act and Retaliation — Can Your Employer Punish You for Asking About a Layoff Notice?
0dev.to Data
WARN Act and Mental Health: What a Layoff Notice Does to Your Wellbeing, and Where to Get Support
0dev.to Data
Dimensionare un pallettizzatore dal lato del progettista: i tre numeri che non stanno nelle brochure
0dev.to Design
How to Choose the Best Stretch Tent Supplier for Your Next Purchase
0dev.to Design
How to spend three slow days in a city you have never visited
0dev.to Productivity