arXiv:2610.36905v1 Announce Type: cross Abstract: This work investigates how site-specific digital twins (DTs) can support wireless localization using simplified distributed infrastructure. We propose PRISM (Probabilistic Refinement using Inverse learning and a Site-specific Model), a DT-guided probabilistic localization scheme with single-antenna access points (APs). Localization is formulated as a probabilistic inverse problem in which a conditional proposal distribution over UE locations is learned from multi-node channel impulse responses (CIRs). The proposal generates candidate locations, with its spread regularized during training and DT consistency used to reweight candidates and refine the estimate at inference. The DT serves as an approximate forward-consistency model, avoiding assumptions of exact channel fidelity or differentiability with respect to user position. Numerical experiments demonstrate gains over data-driven and classical fingerprinting benchmarks, while achieving a median error of approximately $0.7$ m using 10 single-antenna APs. Without retraining, PRISM remains effective under degraded inference-time DT features. PRISM also yields uncertainty estimates from its proposal covariance, with empirical $\chi^2$ calibration approaching ideal behavior as APs increase.