Lecture 010

If we want to use z-transform on continuous distribution like exponential:

\hat{X}(z) = E[z^X] = \int_{t = 0}^\infty z^t \cdot \lambda e^{-\lambda t} dt

This is ugly, but it can be done by express z^t as e^{t \ln z}. But a better method is using Laplace Transform.

Laplace Transform

Laplace Transform: Used for continuous random variable. Laplace Transform of function f is

L_f(s) = \int_0^\infty e^{-st} f(t) dt

For non-negative continuous random variable with density function f_X(t), then Laplace transform is

\tilde{X}(s) = L_{f_X}(S) = \int_0^\infty e^{-st}f_X(t) dt = E[e^{-sX}]

Convergence: \tilde{X}(s) is bounded for any non-negative continuous random variables (since e^{-t} \leq 1 for any non-negative t)

Common Transforms:

Laplace Transform (when it converges) uniquely determines the distribution.

Peeling Onion

\begin{align*} \tilde{X}(s) |_{s = 0} &= 1\\ \tilde{X}'(s) |_{s = 0} &= -E[X]\\ \tilde{X}''(s) |_{s = 0} &= E[X^2]\\ \tilde{X}'''(s) |_{s = 0} &= -E[X^3]\\ \tilde{X}''''(s) |_{s = 0} &= E[X^4]\\ \end{align*}

Note: if s=0 is not defined, you can also use s \rightarrow 0

Example: k-th moment of X \sim \text{Exp}(\lambda)

\begin{align*} \tilde{X}(s) &= \frac{\lambda}{\lambda + s} &= \lambda(\lambda + s)^{-1}\\ \tilde{X}'(s) &= -\lambda(\lambda + s)^{-2} &\implies E[X^1]=\frac{1!}{\lambda^1}\\ \tilde{X}''(s) &= 2\lambda(\lambda + s)^{-3} &\implies E[X^2]=\frac{2!}{\lambda^2}\\ \tilde{X}'''(s) &= -3!\lambda(\lambda + s)^{-4} &\implies E[X^3]=\frac{3!}{\lambda^3}\\ \tilde{X}''''(s) &= 4!\lambda(\lambda + s)^{-5} &\implies E[X^4]=\frac{4!}{\lambda^4}\\ E[X^k] &= \frac{k!}{\lambda^k}\\ \end{align*}

If we define e^{sX} instead of e^{-sX}, we would not need alternating minus sign. However, convergence is no longer guaranteed.

Linearity of Laplace Transforms

X \perp Y \implies \tilde{Z}(s) = \tilde{X}(s) \cdot \tilde{Y}(s) \text{ where } Z = X + Y


In discrete case:

\left[X = \begin{cases} A & \text{with probability } p\\ B & \text{with probability } (1 - p)\\ \end{cases}\right] \implies \tilde{X}(s) = \tilde{A}(s) \cdot p + \tilde{B}(s) \cdot (1 - p)

In continuous case: Let X_Y be a non-negative continuous random variable depends on Y, then

\tilde{X}_Y (s) = \int_{y = 0}^\infty \tilde{X}_y(s) f_Y(y) dy

Summing a Random Number of i.i.d. Random Variable

Recall for discrete X, we have S = \sum_{i = 1}^N X_i \implies \widehat{S}(z) = \widehat{N}\left(\widehat{X}(z)\right) by z-transform.

Let S = \sum_{i = 1}^N X_i where X_i \sim X and N \perp X_i. Then

\tilde{S}(s) = \hat{N}\left(\tilde{X}(s)\right) = \hat{N}\left(z\right)|_{z = \tilde{X}(s)}


\begin{align*} \tilde{S}(s | N = n) =& \left(\tilde{X}(s)\right)^n \tag{by linearity of Laplace transform}\\ \tilde{S}(s) =& \sum_{n = 0}^\infty \tilde{S}(s | N = n) Pr\{N = n\}\\ =& \sum_{n = 0}^\infty \left(\tilde{X}(s)\right)^n Pr\{N = n\}\\ =& \hat{N}(\tilde{X}(s)) \tag{by definition of $N$'s z-transform}\\ \end{align*}

Example: let N \sim \text{Poisson}(\lambda) (therefore \hat{N}(z) = e^{-\lambda(1 - z)}) and X \sim \text{Exp}(\mu) (therefore \tilde{X}(z) = \frac{\mu}{s + \mu})

\tilde{S}(s) = \bar{N}(\tilde{X}(s)) = e^{-\lambda (1 - z) |_{z = \frac{\mu}{z + p}}} = e^{-\frac{\lambda z}{s + \mu}}

Laplace Transform of c.d.f.

Since for continuous random variable, c.d.f. is strongly tight to p.d.f. by integration. We look at Laplace Transform of c.d.f.

\begin{align*} B(x) =& \int_0^x b(t) dt \tag{c.d.f. of b is B}\\ \tilde{b}(s) = L_{b(t)}(s) =& \int_0^\infty e^{-st}b(t) dt \tag{Laplace Transform of b}\\ \tilde{B}(s) = L_{B(t)}(s) = \int_0^\infty e^{-sx}B(x) dx =& \int_0^\infty e^{-sx} \int_0^x b(t) dt dx \tag{Laplace Transform of B}\\ \tilde{B}(s) = \frac{\tilde{b}(s)}{s} \tag{by theorem, see book for proof}\\ \end{align*}

Table of Content