Probability 2 -- Uncertainty Principle
Theorems in this paper mainly comes from lecture notes from EE503 USC.
When hearing about uncertainty principle, people will first think of the famous uncertainty principle theorem in physics. Actually the uncertainty of measurement exists in many different fields. Then a question comes to author's mind, what is the fundamental mathematical structure that leads to these uncertainty relation? Why it shows in different fields like harmonic analysis, statistics and quantum physics? This question is left for further exploration. [[TODO]]
Uncertainty Principle in Physic
It states that there is a limit to the precision with which certain pairs of physical properties, such as position and momentum, can be simultaneously known. -Wikipedia \[ \sigma_x \cdot \sigma_p \ge \frac{\hbar}{2}, \hbar = \frac{h}{2\pi} \]
Uncertainty Principle in Statistics
- The product of variance is greater than the square of covariance. \[ \sigma_X^2 \sigma_Y^2 \ge \sigma_{XY}^2 \]
- The product of sample variance is greater than the square of sample covariance. Here sample variance and covariance are unbiased estimation. \[ S_{XX}^2(n)S_{YY}^2(n) \ge S_{XY}^2(n) \]
- The product of expectation of square is greater than the square of expectation of absolute value of product. \[ E[X^2] E[Y^2] \ge (E[|XY|])^2 \] From these uncertainty principle we directly have \[ \begin{aligned} \rho_{XY} = \frac{\sigma_{XY}}{\sigma_X\sigma_Y} \in [-1, 1] \\ R_{XY}(n) = \frac{S_{XY}(n)}{S_{XX}(n)S_{YY}(n)} \in [-1, 1] \end{aligned} \]
Uncertainty Principle in Harmonic Analysis
A nonzero function and its Fourier transform cannot both be sharply localized.
Variance:
Suppose \(\mu\) is a probability measure defined on \(\mathbb{R}.\) The variance of \(\mu\) is defined as \[ V(\mu) = \inf_{a \in \mathbb{R}} \int(x-a)^2 d\mu(x). \]
Fourier Transform:
Suppose \(f \in L^2(\mathbb{R}),\) the Fourier Transoform is defined as \[ \hat{f}(\xi) = \int e^{-2\pi i \xi x} f(x)dx \]
Heisenberg's Inequality:
If \(f \in L^2(\mathbb{R})\) and \(||f||_2 = 1,\) then \[ V(|f|^2) V(|\hat{f}|^2) \ge \frac{1}{16 \pi^2}. \]Here \(V\) is variance and \(\hat{f}\) is Fourier transformation. In other words, \(\forall f \in L^2(\mathbb{R}), a, b \in \mathbb{R},\) \[ \int (x-a)^2|f(x)|^2 dx \int(\xi-b)^2|\hat{f}(\xi)|^2d\xi \ge \frac{||f||_2^4}{16\pi^2}. \] Another form for the inequality is \[ ||xf||_2 ||\xi \hat{f}|| \ge \frac{||f||_2^2}{4\pi}. \]
cite:
Probability and Statistical Inference, Second Edition, Robert Bartoszynski, Magdalena Niewiadomska-bugaj