Lecture 010
If we want to use z-transform on continuous distribution like exponential:
\hat{X}(z) = E[z^X] = \int_{t = 0}^\infty z^t \cdot \lambda e^{-\lambda t} dt
This is ugly, but it can be done by express z^t as e^{t \ln z}. But a better method is using Laplace Transform.
Laplace Transform
Laplace Transform: Used for continuous random variable. Laplace Transform of function f is
L_f(s) = \int_0^\infty e^{-st} f(t) dt
For non-negative continuous random variable with density function f_X(t), then Laplace transform is
\tilde{X}(s) = L_{f_X}(S) = \int_0^\infty e^{-st}f_X(t) dt = E[e^{-sX}]
Convergence: \tilde{X}(s) is bounded for any non-negative continuous random variables (since e^{-t} \leq 1 for any non-negative t)
Common Transforms:
-
Exponential: \tilde{X}(s) = \int_0^\infty e^{-st}\lambda e^{-\lambda t} dt = \frac{\lambda}{\lambda + s}
-
Constant: \tilde{X}(s) = e^{-sa}
-
Uniform: \tilde{X}(s) = \int_0^\infty e^{-st} \frac{1}{b - a} dt = \frac{e^{-sa} - e^{-sb}}{s(b - a)}
Laplace Transform (when it converges) uniquely determines the distribution.
Peeling Onion
\begin{align*}
\tilde{X}(s) |_{s = 0} &= 1\\
\tilde{X}'(s) |_{s = 0} &= -E[X]\\
\tilde{X}''(s) |_{s = 0} &= E[X^2]\\
\tilde{X}'''(s) |_{s = 0} &= -E[X^3]\\
\tilde{X}''''(s) |_{s = 0} &= E[X^4]\\
\end{align*}
Note: if s=0 is not defined, you can also use s \rightarrow 0
Example: k-th moment of X \sim \text{Exp}(\lambda)
\begin{align*}
\tilde{X}(s) &= \frac{\lambda}{\lambda + s} &= \lambda(\lambda + s)^{-1}\\
\tilde{X}'(s) &= -\lambda(\lambda + s)^{-2} &\implies E[X^1]=\frac{1!}{\lambda^1}\\
\tilde{X}''(s) &= 2\lambda(\lambda + s)^{-3} &\implies E[X^2]=\frac{2!}{\lambda^2}\\
\tilde{X}'''(s) &= -3!\lambda(\lambda + s)^{-4} &\implies E[X^3]=\frac{3!}{\lambda^3}\\
\tilde{X}''''(s) &= 4!\lambda(\lambda + s)^{-5} &\implies E[X^4]=\frac{4!}{\lambda^4}\\
E[X^k] &= \frac{k!}{\lambda^k}\\
\end{align*}
If we define e^{sX} instead of e^{-sX}, we would not need alternating minus sign. However, convergence is no longer guaranteed.
Linearity of Laplace Transforms
X \perp Y \implies \tilde{Z}(s) = \tilde{X}(s) \cdot \tilde{Y}(s) \text{ where } Z = X + Y
Conditioning
In discrete case:
\left[X = \begin{cases}
A & \text{with probability } p\\
B & \text{with probability } (1 - p)\\
\end{cases}\right] \implies \tilde{X}(s) = \tilde{A}(s) \cdot p + \tilde{B}(s) \cdot (1 - p)
In continuous case: Let X_Y be a non-negative continuous random variable depends on Y, then
\tilde{X}_Y (s) = \int_{y = 0}^\infty \tilde{X}_y(s) f_Y(y) dy
Summing a Random Number of i.i.d. Random Variable
Recall for discrete X, we have S = \sum_{i = 1}^N X_i \implies \widehat{S}(z) = \widehat{N}\left(\widehat{X}(z)\right) by z-transform.
Let S = \sum_{i = 1}^N X_i where X_i \sim X and N \perp X_i. Then
\tilde{S}(s) = \hat{N}\left(\tilde{X}(s)\right) = \hat{N}\left(z\right)|_{z = \tilde{X}(s)}
Proof:
\begin{align*}
\tilde{S}(s | N = n) =& \left(\tilde{X}(s)\right)^n \tag{by linearity of Laplace transform}\\
\tilde{S}(s) =& \sum_{n = 0}^\infty \tilde{S}(s | N = n) Pr\{N = n\}\\
=& \sum_{n = 0}^\infty \left(\tilde{X}(s)\right)^n Pr\{N = n\}\\
=& \hat{N}(\tilde{X}(s)) \tag{by definition of $N$'s z-transform}\\
\end{align*}
Example: let N \sim \text{Poisson}(\lambda) (therefore \hat{N}(z) = e^{-\lambda(1 - z)}) and X \sim \text{Exp}(\mu) (therefore \tilde{X}(z) = \frac{\mu}{s + \mu})
\tilde{S}(s) = \bar{N}(\tilde{X}(s)) = e^{-\lambda (1 - z) |_{z = \frac{\mu}{z + p}}} = e^{-\frac{\lambda z}{s + \mu}}
Laplace Transform of c.d.f.
Since for continuous random variable, c.d.f. is strongly tight to p.d.f. by integration. We look at Laplace Transform of c.d.f.
\begin{align*}
B(x) =& \int_0^x b(t) dt \tag{c.d.f. of b is B}\\
\tilde{b}(s) = L_{b(t)}(s) =& \int_0^\infty e^{-st}b(t) dt \tag{Laplace Transform of b}\\
\tilde{B}(s) = L_{B(t)}(s) = \int_0^\infty e^{-sx}B(x) dx =& \int_0^\infty e^{-sx} \int_0^x b(t) dt dx \tag{Laplace Transform of B}\\
\tilde{B}(s) = \frac{\tilde{b}(s)}{s} \tag{by theorem, see book for proof}\\
\end{align*}