Gaussian Annulus Theorem
Contents
Gaussian Annulus Theorem¶
For a d-dimensional spherical Gaussian with unit variance in each direction, for any β ≤ √d, $\(3e^{-cβ^{2}}\)$ all but at most of the probability mass lies within the annulus √d-β ≤ |x| ≤ √d+β, where c is a fixed positive constant.
Why does this happen?¶
Intention: Consider annuali of this thickness at increasing distance from centre.
The value of pdf decreases with increasing radii
The value of the annulus increases exponentially with increasing radii
Relation between two samples¶
Suppose we sample \({x_i,x_j}\) both from N(0,I)— what is the angle between them?
Angle between two samples¶
The previous analysis tells us for \( x.y ~ N(0,\) \({I_d}) \) \( {||x-y||^2 = 2d ± θ(√d)} \)
\( {||x||^2 + ||y||^2 - 2 {X^T}y} = {2d ± θ(√d)}\)
We also saw \( {||x||^2 , ||y||^2 = {d ± θ(√d)}}\)
\( X^Ty = O(√d)\)
\( cos(<(x,y)>) = \frac{O(√d)}{θ(√d)} = O(d^\frac{-1}{2})\)
x is almost orthogonal to y