AI & ML
Fractional is Better: Learnable Derivative Orders in Neural Operator Learning
arXiv stat.ML0 views
arXiv:2610.20363v1 Announce Type: cross Abstract: Neural operators learn mappings between function spaces, enabling fast surrogate solutions to partial differential equations. Despite remarkable architectural diversity, these methods often share a common input representation: raw coordinate-value pairs $(x, u(x))$. We ask whether inputs aligned with PDE differential structure can improve learning. Through Picard iteration on mild solutions, we show that derivatives of the input appear explicitly in the solution operator, suggesting that providing derivative features should reduce the network's implicit differentiation burden. We prove this intuition: providing derivative features improves approximation bounds from $O(W^{-(s-m)/d})$ to $O(W^{-s/d})$, where $W$ is parameter count, $s$ is input regularity, $m$ is the PDE order, and $d$ is spatial dimension. Our central finding, however, is a surprise: the optimal derivative order $\beta^*$ is strictly less than the PDE order $m$. This gap arises from a bias-variance tradeoff in spectral space that we characterize in closed form. Learning $\beta$ from data achieves automatic spectral regularization. We introduce $\partial$-NO (del-NO), for derivative-augmented neural operators, an architecture-agnostic augmentation that provides learnable fractional derivative features to any neural operator backbone. Across benchmark problems and architectures, $\partial$-NO consistently improves prediction accuracy, with learned orders that reflect a representation of known physics modulated by noise and finite-sample constraints.
Source:https://arxiv.org/abs/2610.20363PreviousBeyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE QuantizationNextWhen Does Momentum Help in Reinforcement Learning with Verifiable Rewards?
Related posts
How to Turn a Process Document Into a Working App With AI
0dev.to Machine Learning
Which AI subscription should I buy?
0dev.to Machine Learning
Your Classifier Says 0.97. Six Ways That Number Lies in .NET 10
0dev.to Machine Learning
Which AI song generator fits a lyrics-first workflow?
0dev.to Tutorial
Anima 2.9B in ComfyUI: Six Workflows, From Official T2I to Tiled Redraw and Smart Img2Img
0dev.to Tutorial
How to calculate how many cartons fit on a pallet (with a worked example)
0dev.to Tutorial