# Lecture 008

## Normal Distribution

### Definition

Normal Distribution: $X \sim \text{Normal}(\mu, \sigma^2)$ or $X \sim \text{Gaussian}(\mu, \sigma^2)$

\begin{align} f_X(x) =& \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}(\frac{x - \mu}{\sigma})^2}\\ E[X] =& \mu\\ Var(X) =& \sigma^2\\ \end{align}

where $\mu$ is mean and $\sigma$ is standard deviation.

Standard Normal: $X \sim \text{Normal}(0, 1)$

\begin{align} f_X(x) =& \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}x^2}\\ E[X] =& 0\\ Var(X) =& 1\\ \Phi = F_X(x) =& \frac{1}{\sqrt{2\pi}}\int_{-\infty}^x e^{-\frac{t^2}{2}}dt\\ \end{align}

### Linear Transform Property

Linear Transform Property: for $X \sim \text{Normal}(\mu, \sigma^2)$, then

Y = aX + b \sim \text{Normal}(a\mu + b, a^2 \sigma^2)

Proof:

\begin{align*} \frac{d}{dy}F_Y(y) =& \frac{d}{dy}F_X(\frac{y - b}{a}) \tag{derivative of both side}\\ \frac{d}{dy} \int_{-\infty}^\infty f_Y(t)dt =& \frac{d}{dy} \int_{-\infty}^{\frac{y - k}{a}}f_X(t)dt\\ f_Y(y) =& f_X(\frac{y - b}{a}) \cdot \frac{d}{dy}(\frac{y - b}{a}) \tag{by Fundamental Theorem of Calculus}\\ =& f_X(\frac{y - b}{a}) \cdot \frac{1}{a}\\ =& \frac{1}{a \sqrt{2\pi} \sigma}e^{-(\frac{y - b}{a} - \mu)^2 / 2 \sigma^2}\\ =& \frac{1}{a \sqrt{2\pi} \sigma}e^{-(y - (b + a\mu))^2 / 2 a^2 \sigma^2}\\ \end{align*}

Be careful with notation involving continuous random variables, since probability of equal is zero: - WRONG: $f_Y = Pr\{Y = y\} = Pr\{aX + b - y\} = Pr\{X = \frac{y - b}{a}\} = f_X(\frac{y - b}{a})$ - CORRECT: $F_Y(y) = Pr\{Y < y\} = Pr\{aX + b \leq y\} = Pr\{X \leq \frac{y - b}{a}\} = F_X(\frac{y - b}{a})$

### c.d.f. Table

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009
0 0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359
0.1 0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753
0.2 0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141
0.3 0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517
0.4 0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6808 0.6844 0.6879
0.5 0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224
0.6 0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549
0.7 0.7580 0.7611 0.7642 0.7673 0.7704 0.7734 0.7764 0.7794 0.7823 0.7852
0.8 0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133
0.9 0.8159 0.8186 0.8212 0.8238 0.8264 0.8289 0.8315 0.8340 0.8365 0.8389
1 0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621
1.1 0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830
1.2 0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015
1.3 0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177
1.4 0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319
1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441
1.6 0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545
1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633
1.8 0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706
1.9 0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767
2 0.9772 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817
2.1 0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857
2.2 0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890
2.3 0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916
2.4 0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932 0.9934 0.9936
2.5 0.9938 0.9940 0.9941 0.9943 0.9945 0.9946 0.9948 0.9949 0.9951 0.9952
2.6 0.9953 0.9955 0.9956 0.9957 0.9959 0.9960 0.9961 0.9962 0.9963 0.9964
2.7 0.9965 0.9966 0.9967 0.9968 0.9969 0.9970 0.9971 0.9972 0.9973 0.9974
2.8 0.9974 0.9975 0.9976 0.9977 0.9977 0.9978 0.9979 0.9979 0.9980 0.9981
2.9 0.9981 0.9982 0.9982 0.9983 0.9984 0.9984 0.9985 0.9985 0.9986 0.9986
3 0.9987 0.9987 0.9987 0.9988 0.9988 0.9989 0.9989 0.9989 0.9990 0.9990

Convert to Standard Normal:

X \sim \text{Normal}(\mu, \sigma^2) \iff Y = \frac{X - \mu}{\sigma} \sim \text{Normal}(0, 1)

Then the probability that $X$ deviates from its mean by $k$ standard deviation is

Pr\{-k\sigma < X < k \sigma\} = Pr\{-k < \frac{X - \mu}{\sigma} < k\} = Pr\{-k < Y < k\}

The result of above formula is: for any normal distribution, the deviation by $x$ standard deviation is the same as standard normal.

### Central Limit Theorem

Let $S_n = X_1 + X_2 + ... + X_n = \sum_{i = 0}^n X_i$ where each $X_i$ is independent and identically distributed with mean $\mu$ and variance $\sigma^2$. Then

1. $E[S_n] = n \mu$
2. $Var(S_n) = n \sigma^2$
3. $Z_n = \frac{S_a - E[S_n]}{\sqrt{Var(S_n)}} = \frac{S_a - n \mu}{\sigma \sqrt{n}}, \lim_{n \rightarrow \infty} Z_n = \text{Normal}(0, 1)$
4. Central Limit:
(\forall z)(\lim_{n \rightarrow \infty} Pr\{Z_n \leq z\} = \Phi(z) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^z e^{-x^2/2}dx)
1. Application:
Pr\{a < S_n < b\} = Pr\{\frac{a - E[S_n]}{\sqrt{Var(S_n)}} < \frac{S_n - E[S_n]}{\sqrt{Var(S_n)}} < \frac{b - E[S_n]}{\sqrt{Var(S_n)}}\}

Note that $S_n$ itself does not approach Normal Distribution because 1. is not well defined when $n \rightarrow \infty$ and therefore will have infinite mean and variance. 2. sum of discrete random variable is still discrete

We can prove $\text{Binomial}(n, p)$ approach to Normal Distribution since it is sum of Bernoulli.

Table of Content