AI & ML
A Matching Lower Bound for Online Smooth Calibration
arXiv stat.ML0 views
arXiv:2611.16097v2 Announce Type: replace-cross Abstract: We study smooth calibration error in sequential binary prediction, a measure of how far probabilistic forecasts are from being calibrated that is robust to small perturbations of the forecasts. The minimax smooth calibration error is known to lie between $\Omega(T^{1/3})$ and $O(\sqrt T)$(Qiao and Zheng, 2024), where $T$ is the number of rounds. We close this gap for both adaptive and oblivious adversaries. We construct an adaptive adversary whose outcome mean moves with the forecaster's cumulative residual, which makes compensating for past residuals costly and forces every forecaster to incur $\Omega(\sqrt T)$ expected smooth calibration error. Simulating this adversary against the forecaster's average prediction yields an oblivious adversary with the same guarantee. These lower bounds match the known $O(\sqrt T)$ upper bound up to universal constants, and are therefore minimax optimal. Our results also imply matching $\Theta(\sqrt T)$ bounds for calibration distance and lower calibration distance. The proof ideas are partially developed by GPT-6 Astra under the authors' guidance.
Source:https://arxiv.org/abs/2611.16097PreviousHow Path Integral Diffusion Decides a Finite Spin GlassNextfOGA: An Orthogonal Greedy Algorithm for Fractional Laplacian Problems
Related posts
Eval-Driven Development: How to Test AI Features Beyond Traditional Unit Tests
0dev.to Testing

A RAG answer can cite every sentence and still be wrong: a script to reproduce it
0dev.to Testing
![Vision Language Action Models — Ch.09: Generalist Robot Policies [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiu9r3c6ct01wklmuk30s.png)
Vision Language Action Models — Ch.09: Generalist Robot Policies [EN]
0dev.to Programming
![Vision Language Action Models — Ch.08: Humanoid VLA Architectures [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwvvm5efdgcglkt54bd4u.png)
Vision Language Action Models — Ch.08: Humanoid VLA Architectures [EN]
0dev.to Programming
![Vision Language Action Models — Ch.06: Language to Action Pipelines [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwnqy6w0otpzb0uvivwn8.png)
Vision Language Action Models — Ch.06: Language to Action Pipelines [EN]
0dev.to Programming
![Vision Language Action Models — Ch.05: Vision Language Models [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbw5a20v9raz6duiimhzg.png)
Vision Language Action Models — Ch.05: Vision Language Models [EN]
0dev.to Programming