AI & ML
Differentially Private Score Estimation with Applications to Score-Based Generative Models
arXiv stat.TH0 views
arXiv:2611.25536v1 Announce Type: cross Abstract: Estimating the score function of a distribution plays a central role in statistics and machine learning, with applications in distribution estimation, sampling, and score-based generative modeling. However, learning scores from sensitive data raises privacy concerns. These concerns are particularly pronounced in score-based generative modeling, where trained models may memorize individual training examples and expose sensitive information through membership inference and data extraction attacks. Although non-private score estimation has been extensively studied, the statistical limits of its private counterpart remain less understood. In this work, we study score estimation under $(\alpha,\beta)$-differential privacy for $d$-dimensional sub-Gaussian distributions with locally $\gamma$-H\"older-continuous scores. We develop a sampling-based private score estimator using quadratic B-splines and the exponential mechanism, and we implement our proposal through projected Langevin Monte Carlo with runtime polynomial in the sample size $n$. The proposed estimator satisfies $(\alpha,\beta)$-differential privacy, and achieves expected density-weighted squared error $\mbox{polylog}(n) \max(n^{-2\gamma/(d+2\gamma+2)},\,(n\alpha)^{-2\gamma/(d+\gamma+1)})$. We also establish a nearly matching private minimax lower bound, thereby characterizing the privacy-accuracy trade-off for score estimation. As an application, we construct a differentially private diffusion model whose privacy guarantee is independent of the number of denoising steps. We show that, with an appropriate discretization scheme, the resulting diffusion model attains nearly minimax optimal expected total variation error among all private distribution estimators. We validate our theoretical findings through experiments on both synthetic and real-world datasets.
Source:https://arxiv.org/abs/2611.25536PreviousImproved Constants in the Kautz-Singleton Construction for Probabilistic Group TestingNextAdaptive Tests of Pricing Restrictions with Observed and Latent Factors
Related posts
Trying to get our Arms around AI efficiency
0Stack Overflow Blog
Why an Embedded Game Fails on Some Phones, and How to Debug It
0dev.to
Microsoft 365 vs Office 2024: The Break-Even Math
0dev.to
Master AI Evals: Build Reliable, Scalable Model Evaluation Pipelines
0dev.to
What to give someone who misses a place
0dev.to
Google is pushing OEMs against GrapheneOS and bootloader unlocking
0Hacker News