Lecture 008

Normal Distribution

Definition

Normal Distribution: X \sim \text{Normal}(\mu, \sigma^2) or X \sim \text{Gaussian}(\mu, \sigma^2)

\begin{align} f_X(x) =& \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}(\frac{x - \mu}{\sigma})^2}\\ E[X] =& \mu\\ Var(X) =& \sigma^2\\ \end{align}

where \mu is mean and \sigma is standard deviation.

Standard Normal: X \sim \text{Normal}(0, 1)

\begin{align} f_X(x) =& \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}x^2}\\ E[X] =& 0\\ Var(X) =& 1\\ \Phi = F_X(x) =& \frac{1}{\sqrt{2\pi}}\int_{-\infty}^x e^{-\frac{t^2}{2}}dt\\ \end{align}

Linear Transform Property

Linear Transform Property: for X \sim \text{Normal}(\mu, \sigma^2), then

Y = aX + b \sim \text{Normal}(a\mu + b, a^2 \sigma^2)

Proof:

\begin{align*} \frac{d}{dy}F_Y(y) =& \frac{d}{dy}F_X(\frac{y - b}{a}) \tag{derivative of both side}\\ \frac{d}{dy} \int_{-\infty}^\infty f_Y(t)dt =& \frac{d}{dy} \int_{-\infty}^{\frac{y - k}{a}}f_X(t)dt\\ f_Y(y) =& f_X(\frac{y - b}{a}) \cdot \frac{d}{dy}(\frac{y - b}{a}) \tag{by Fundamental Theorem of Calculus}\\ =& f_X(\frac{y - b}{a}) \cdot \frac{1}{a}\\ =& \frac{1}{a \sqrt{2\pi} \sigma}e^{-(\frac{y - b}{a} - \mu)^2 / 2 \sigma^2}\\ =& \frac{1}{a \sqrt{2\pi} \sigma}e^{-(y - (b + a\mu))^2 / 2 a^2 \sigma^2}\\ \end{align*}

Be careful with notation involving continuous random variables, since probability of equal is zero: - WRONG: f_Y = Pr\{Y = y\} = Pr\{aX + b - y\} = Pr\{X = \frac{y - b}{a}\} = f_X(\frac{y - b}{a}) - CORRECT: F_Y(y) = Pr\{Y < y\} = Pr\{aX + b \leq y\} = Pr\{X \leq \frac{y - b}{a}\} = F_X(\frac{y - b}{a})

c.d.f. Table

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009
0 0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359
0.1 0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753
0.2 0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141
0.3 0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517
0.4 0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6808 0.6844 0.6879
0.5 0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224
0.6 0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549
0.7 0.7580 0.7611 0.7642 0.7673 0.7704 0.7734 0.7764 0.7794 0.7823 0.7852
0.8 0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133
0.9 0.8159 0.8186 0.8212 0.8238 0.8264 0.8289 0.8315 0.8340 0.8365 0.8389
1 0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621
1.1 0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830
1.2 0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015
1.3 0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177
1.4 0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319
1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441
1.6 0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545
1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633
1.8 0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706
1.9 0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767
2 0.9772 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817
2.1 0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857
2.2 0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890
2.3 0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916
2.4 0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932 0.9934 0.9936
2.5 0.9938 0.9940 0.9941 0.9943 0.9945 0.9946 0.9948 0.9949 0.9951 0.9952
2.6 0.9953 0.9955 0.9956 0.9957 0.9959 0.9960 0.9961 0.9962 0.9963 0.9964
2.7 0.9965 0.9966 0.9967 0.9968 0.9969 0.9970 0.9971 0.9972 0.9973 0.9974
2.8 0.9974 0.9975 0.9976 0.9977 0.9977 0.9978 0.9979 0.9979 0.9980 0.9981
2.9 0.9981 0.9982 0.9982 0.9983 0.9984 0.9984 0.9985 0.9985 0.9986 0.9986
3 0.9987 0.9987 0.9987 0.9988 0.9988 0.9989 0.9989 0.9989 0.9990 0.9990

Convert to Standard Normal:

X \sim \text{Normal}(\mu, \sigma^2) \iff Y = \frac{X - \mu}{\sigma} \sim \text{Normal}(0, 1)

Then the probability that X deviates from its mean by k standard deviation is

Pr\{-k\sigma < X < k \sigma\} = Pr\{-k < \frac{X - \mu}{\sigma} < k\} = Pr\{-k < Y < k\}

The result of above formula is: for any normal distribution, the deviation by x standard deviation is the same as standard normal.

Central Limit Theorem

Let S_n = X_1 + X_2 + ... + X_n = \sum_{i = 0}^n X_i where each X_i is independent and identically distributed with mean \mu and variance \sigma^2. Then

  1. E[S_n] = n \mu
  2. Var(S_n) = n \sigma^2
  3. Z_n = \frac{S_a - E[S_n]}{\sqrt{Var(S_n)}} = \frac{S_a - n \mu}{\sigma \sqrt{n}}, \lim_{n \rightarrow \infty} Z_n = \text{Normal}(0, 1)
  4. Central Limit:
(\forall z)(\lim_{n \rightarrow \infty} Pr\{Z_n \leq z\} = \Phi(z) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^z e^{-x^2/2}dx)
  1. Application:
Pr\{a < S_n < b\} = Pr\{\frac{a - E[S_n]}{\sqrt{Var(S_n)}} < \frac{S_n - E[S_n]}{\sqrt{Var(S_n)}} < \frac{b - E[S_n]}{\sqrt{Var(S_n)}}\}

Note that S_n itself does not approach Normal Distribution because 1. is not well defined when n \rightarrow \infty and therefore will have infinite mean and variance. 2. sum of discrete random variable is still discrete

We can prove \text{Binomial}(n, p) approach to Normal Distribution since it is sum of Bernoulli.

Table of Content