Nearest-Point Projection Removes the Noise Singularity in Manifold Score Matching

When data live on a curved surface, ordinary denoising score matching can get tangled in a noise channel whose variance blows up like d/σ^2 as the corruption level σ shrinks. This paper shows a clean way out: condition on the nearest-point projection π(X), and the singularity disappears. The resulting conditional expectation is the unique L2-optimal Rao-Blackwellized predictor among all estimators that depend only on the projected observation. The authors then expand this canonical target for small noise and show that it matches the intrinsic Riemannian score up to an explicit order-σ^2 correction. That correction splits into an intrinsic Tweedie term and an extrinsic curvature term involving the Weingarten and Ricci operators. In the flat case, the method becomes ordinary lower-dimensional Gaussian denoising score matching exactly. On the sphere S^d, the extrinsic piece simplifies to (1−d/2)∇_M log q, and it cancels on S^2 even though the intrinsic Tweedie term remains.

On a curved surface, tiny amounts of noise can make a score target swell like d/σ² as σ shrinks. That is not a cosmetic glitch; it means the signal you are trying to learn picks up a hidden channel of variance from the normal direction, so the cleaner the data look, the harder the regression can become. If you've ever tried to recover a shape from a blurred photograph, the trap is familiar: the blur outside the object still changes what the model thinks is inside it. This work answers that by asking a very plain question: what happens if the model listens only after each noisy point is snapped back to its nearest point on the surface?

The singularity disappears when you keep only the projection

The answer is surprisingly clean. Once the noisy observation X is replaced by its nearest-point projection π(X), the conditional expectation of the tangent DSM target becomes the unique L2-optimal Rao-Blackwellized predictor among all estimators that depend only on π(X). In plain English, the off-surface jitter no longer gets to bully the estimate. The small-noise expansion then says this canonical target is not just close to the intrinsic Riemannian score; it equals that score up to an explicit order-σ² correction. That correction splits into two parts: an intrinsic Tweedie term and an extrinsic curvature term built from the Weingarten and Ricci operators. On a flat manifold, the whole construction collapses exactly to ordinary lower-dimensional Gaussian DSM. On S^d, the extrinsic piece simplifies to (1 - d/2)∇_M log q, and it vanishes on S^2 even though the intrinsic Tweedie term stays.

How averaging over π(X) does the hard part

The trick is not to invent a new manifold diffusion. It is to keep the usual ambient Gaussian corruption, then throw away the part of the noisy sample that lies in the normal fiber once the point has been projected back to the surface. That projection acts like a geometric filter: it keeps the information the manifold can actually support and averages out the singular channel that came from drifting off the surface. Rao-Blackwellization does the rest. Because the predictor depends only on π(X), and because conditional expectation is the best mean-square choice among such predictors, the result is canonical rather than heuristic. The expansion around small noise then shows how geometry enters twice — first through the intrinsic score itself, and then through curvature, which bends the correction away from the flat-space case.

Why this bridge matters

That distinction matters because it separates two jobs that often get blurred together. Ambient Gaussian noise can stay in place, so the familiar DSM pipeline does not have to be abandoned, but the learned target is no longer a mysterious surrogate. It becomes a principled bridge from Euclidean corruption to intrinsic geometry. In the flat case, the bridge is exact. On spheres, the bridge has a simple extra term, and on S^2 that extra term drops out completely. So this result does more than tidy up notation: it explains when ordinary ambient DSM is quietly learning the intrinsic score, when it is learning a corrected version, and which part of that correction comes from curvature rather than noise.

The sphere makes the edge case visible

The sharpest stress test is the spherical case beyond S^2, where the extrinsic correction survives as (1 - d/2)∇_M log q instead of disappearing. That makes S^d with d ≠ 2 a useful proving ground: if the nearest-point projection really carries the right information, the correction should track that simple factor instead of turning messy. The surprise in this story is that the dangerous part of the noise is not the part you keep; it is the part you average away. Once that clicks, the open challenge is no longer whether ambient DSM can speak geometry, but how faithfully it does so on curved spaces where the curvature terms refuse to cancel.