Lecture 010
If we want to use ztransform on continuous distribution like exponential:
\hat{X}(z) = E[z^X] = \int_{t = 0}^\infty z^t \cdot \lambda e^{\lambda t} dt
This is ugly, but it can be done by express z^t as e^{t \ln z}. But a better method is using Laplace Transform.
Laplace Transform
Laplace Transform: Used for continuous random variable. Laplace Transform of function f is
L_f(s) = \int_0^\infty e^{st} f(t) dt
For nonnegative continuous random variable with density function f_X(t), then Laplace transform is
\tilde{X}(s) = L_{f_X}(S) = \int_0^\infty e^{st}f_X(t) dt = E[e^{sX}]
Convergence: \tilde{X}(s) is bounded for any nonnegative continuous random variables (since e^{t} \leq 1 for any nonnegative t)
Common Transforms:

Exponential: \tilde{X}(s) = \int_0^\infty e^{st}\lambda e^{\lambda t} dt = \frac{\lambda}{\lambda + s}

Constant: \tilde{X}(s) = e^{sa}

Uniform: \tilde{X}(s) = \int_0^\infty e^{st} \frac{1}{b  a} dt = \frac{e^{sa}  e^{sb}}{s(b  a)}
Laplace Transform (when it converges) uniquely determines the distribution.
Peeling Onion
\begin{align*}
\tilde{X}(s) _{s = 0} &= 1\\
\tilde{X}'(s) _{s = 0} &= E[X]\\
\tilde{X}''(s) _{s = 0} &= E[X^2]\\
\tilde{X}'''(s) _{s = 0} &= E[X^3]\\
\tilde{X}''''(s) _{s = 0} &= E[X^4]\\
\end{align*}
Note: if s=0 is not defined, you can also use s \rightarrow 0
Example: kth moment of X \sim \text{Exp}(\lambda)
\begin{align*}
\tilde{X}(s) &= \frac{\lambda}{\lambda + s} &= \lambda(\lambda + s)^{1}\\
\tilde{X}'(s) &= \lambda(\lambda + s)^{2} &\implies E[X^1]=\frac{1!}{\lambda^1}\\
\tilde{X}''(s) &= 2\lambda(\lambda + s)^{3} &\implies E[X^2]=\frac{2!}{\lambda^2}\\
\tilde{X}'''(s) &= 3!\lambda(\lambda + s)^{4} &\implies E[X^3]=\frac{3!}{\lambda^3}\\
\tilde{X}''''(s) &= 4!\lambda(\lambda + s)^{5} &\implies E[X^4]=\frac{4!}{\lambda^4}\\
E[X^k] &= \frac{k!}{\lambda^k}\\
\end{align*}
If we define e^{sX} instead of e^{sX}, we would not need alternating minus sign. However, convergence is no longer guaranteed.
Linearity of Laplace Transforms
X \perp Y \implies \tilde{Z}(s) = \tilde{X}(s) \cdot \tilde{Y}(s) \text{ where } Z = X + Y
Conditioning
In discrete case:
\left[X = \begin{cases}
A & \text{with probability } p\\
B & \text{with probability } (1  p)\\
\end{cases}\right] \implies \tilde{X}(s) = \tilde{A}(s) \cdot p + \tilde{B}(s) \cdot (1  p)
In continuous case: Let X_Y be a nonnegative continuous random variable depends on Y, then
\tilde{X}_Y (s) = \int_{y = 0}^\infty \tilde{X}_y(s) f_Y(y) dy
Summing a Random Number of i.i.d. Random Variable
Recall for discrete X, we have S = \sum_{i = 1}^N X_i \implies \widehat{S}(z) = \widehat{N}\left(\widehat{X}(z)\right) by ztransform.
Let S = \sum_{i = 1}^N X_i where X_i \sim X and N \perp X_i. Then
\tilde{S}(s) = \hat{N}\left(\tilde{X}(s)\right) = \hat{N}\left(z\right)_{z = \tilde{X}(s)}
Proof:
\begin{align*}
\tilde{S}(s  N = n) =& \left(\tilde{X}(s)\right)^n \tag{by linearity of Laplace transform}\\
\tilde{S}(s) =& \sum_{n = 0}^\infty \tilde{S}(s  N = n) Pr\{N = n\}\\
=& \sum_{n = 0}^\infty \left(\tilde{X}(s)\right)^n Pr\{N = n\}\\
=& \hat{N}(\tilde{X}(s)) \tag{by definition of $N$'s ztransform}\\
\end{align*}
Example: let N \sim \text{Poisson}(\lambda) (therefore \hat{N}(z) = e^{\lambda(1  z)}) and X \sim \text{Exp}(\mu) (therefore \tilde{X}(z) = \frac{\mu}{s + \mu})
\tilde{S}(s) = \bar{N}(\tilde{X}(s)) = e^{\lambda (1  z) _{z = \frac{\mu}{z + p}}} = e^{\frac{\lambda z}{s + \mu}}
Laplace Transform of c.d.f.
Since for continuous random variable, c.d.f. is strongly tight to p.d.f. by integration. We look at Laplace Transform of c.d.f.
\begin{align*}
B(x) =& \int_0^x b(t) dt \tag{c.d.f. of b is B}\\
\tilde{b}(s) = L_{b(t)}(s) =& \int_0^\infty e^{st}b(t) dt \tag{Laplace Transform of b}\\
\tilde{B}(s) = L_{B(t)}(s) = \int_0^\infty e^{sx}B(x) dx =& \int_0^\infty e^{sx} \int_0^x b(t) dt dx \tag{Laplace Transform of B}\\
\tilde{B}(s) = \frac{\tilde{b}(s)}{s} \tag{by theorem, see book for proof}\\
\end{align*}