Backend
Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods
arXiv math.IT0 views
arXiv:2609.31107v2 Announce Type: replace-cross Abstract: We study Bayesian optimization (BO) through the lens of information geometry. Pulling back the Fisher information metric through the surrogate posterior map yields a local sensitivity tensor on the input space, which leads to an upper bound on the gradient of reparameterizable acquisition functions. This view explains vanishing-gradient behavior in high-dimensional BO and provides a common interpretation of heuristics such as RAASP and dimension-scaled lengthscales. Building on this analysis, we propose FITR, a trust-region-based BO method that replaces lengthscale-based scaling by local pullback-Fisher weights. FITR is not restricted to GP kernels with explicit lengthscales. On GP benchmarks with an SE kernel, experiments show competitive performance using FITR. The proposed method also easily generalizes to non-isotropic surrogates, although the gains are more task-dependent in that setting.
Source:https://arxiv.org/abs/2609.31107PreviousA sharp analysis of Root-MUSIC: locations of correct and extraneous rootsNextWhen the Persona Contradicts the Line: Diagnosing Plan-to-Synthesis Failures in Chain-of-Thought TTS
Related posts
The Building Is a System: What Software Engineers Can Learn from Construction Site Failures
0dev.to

Part 4: When the Screen Changes Silently: Live Regions, Headings and Focus in Compose
0dev.to
FlatCake: Why Tiered KV Cache Storage Is Holding LLM Serving Back
0dev.to
Why Scrum is Not a Silver Bullet: 5 Structural Drawbacks and Fixes
0dev.to
Agentic Frameworks Compared: LangGraph, CrewAI & More
0dev.to
The Real-World Pain
0dev.to