So you have X_1, X_2, ... random variables with the same distribution. And N discrete, positive, integer random variable with X_i \perp N

Let us compute S = \sum_{i = 1}^N X_i

Theorem: Let X_1, X_2, ... be i.i.d. (Independent and identically distributed) random variables, where X_i \sim X. Let S = \sum_{i = 1}^N X_i where N \perp X_i, then:

\begin{align}
E[S] =& E[N]E[X]\\
E[S^2] =& E[N]Var(X) + E[N^2]E[X]^2\\
Var(S) =& E[N]Var(X) + E[X]^2Var(N)\\
\end{align}

\begin{align*}
E[S] =& E[\sum_{i = 1}^N X_i]\\
=& \sum_{n = 1}^\infty E[\sum_{i = 1}^N X_i | N = n] \cdot Pr\{N = n\}\\
=& \sum_{n = 1}^\infty E[\sum_{i = 1}^n X_i | N = n] \cdot Pr\{N = n\}\\
=& \sum_{n = 1}^\infty E[\sum_{i = 1}^n X_i] \cdot Pr\{N = n\} \tag{by $X_i, N$ independent}\\
=& \sum_{n = 1}^\infty nE[X_i] \cdot Pr\{N = n\}\\
=& E[X_i]\sum_{n = 1}^\infty n \cdot Pr\{N = n\}\\
=& E[X_i]E[N]\\
\end{align*}

You want to compute Var(S) = E[S^2] - E[S]^2. We now compute E[S^2]

\begin{align*}
&E[S^2]\\
=& E[(\sum_{i = 1}^N X_i)^2]\\
=& \sum_n E[(\sum_{i = 1}^N X_i)^2 | N = n] Pr\{N = n\}\\
=& \sum_n E[(\sum_{i = 1}^n X_i)^2] Pr\{N = n\}\\
=& \sum_n E[(X_1 + X_2 + ... + X_n)^2] Pr\{N = n\}\\
=& \sum_n nE[X_1^2] + (n^2 - n)E[X_1X_2] Pr\{N = n\}\tag{Note that $X_1 \not\perp X_1$, $X_1 \perp X_2$}\\
=& \sum_n nE[X^2] Pr\{N = n\} + \sum_n (n^2 - n)E[X]^2 Pr\{N = n\}\\
=& E[N]E[X^2] + E[N^2]E[X]^2 - E[N]E[X]^2\\
=& E[N](E[X^2] - E[X]^2) + E[N^2]E[X]^2\\
=& E[N]Var(X) + E[N^2]E[X]^2\\
\end{align*}

With above E[S^2] = E[N]Var(X) + E[N^2]E[X]^2:

\begin{align*}
Var(S) =& E[S^2] - E[X]^2\\
=& E[N]Var(X) + E[N^2]E[X]^2 - E[X]^2\\
=& E[N]Var(X) + E[N^2]E[X]^2 - E[X]^2E[N]^2 \tag{by $E[S] = E[X]E[N]$ above}\\
=& E[N]Var(X) + E[X]^2(E[N^2] - E[N]^2)\\
=& E[N]Var(X) + E[X]^2Var(N)\\
\end{align*}

We have time t = 0, 1, 2, 3..., and start from a node. On each step, with probability \frac{1}{2}, a leaf will stay inert, with probability \frac{1}{2}, a leaf will split to 2.

Define X_t = \text{number of leaves in time} = t, then we see:

Define Y_i = \begin{cases} 1 & \text{with half probability}\\ 2 & \text{with half probability}\\ \end{cases}

\begin{align*}
X_t =& \sum_{i = 1}^{X_{t - 1}} Y_i\\
=& \sum_{i = 1}^{X_{t - 1}} 1 \cdot Pr\{\text{this leaf stay inert}\} + 2 \cdot Pr\{\text{this leaf split to 2}\}\\
\end{align*}

We can easily calculate:

- E[Y] = \frac{3}{2}
- E[Y^2] = 1^2 \frac{1}{2} + 2^2 \frac{1}{2} = \frac{5}{2}
- Var(Y) = \frac{10}{4} - \frac{9}{4} = \frac{1}{4}

Now using the formula we calculate:

\begin{align*}
&E[X_t]\\
=& E[X_{t - 1}] \cdot E[Y]\\
=& E[X_{t - 1}](\frac{3}{2})\\
=& (\frac{3}{2})^{y - 1}(\frac{3}{2}) \tag{by unfolding $E[X_{t - 1}]$}\\
=& (\frac{3}{2})^y\\
\end{align*}

\begin{align*}
&Var(X_t)\\
=& E[X_{t - 1}]Var(Y) + E[Y]^2Var(X_{t - 1})\\
=& (\frac{3}{2})^{t - 1} \cdot \frac{1}{4} + (\frac{3}{2})^2 Var(X_{t - 1})\\
\end{align*}

Stochastically Dominate: X stochastically dominates Y (write as X \geq_{st} Y) if (\forall i)(Pr\{X > i\} \geq Pr\{Y > i\})

E[X^2] \geq E[X]^2

E[X^s] \geq E[X]^s (\text{for integer } x \geq 2)

Definition of convex function: (all point on convex function sit above a line that pass through it) A real-valued function g(x) on interval S \subseteq \mathbb{R} is convex on S if:

- g(t) = at + b
- (\forall x \in S)(g(x) \geq y(x) \equiv ax + b) where the line y(x) = ax + b is called a supporting line for the function g at t.

Theorem: Let X be a random variable that takes on values in an interval S and let g : S \rightarrow \mathbb{R} be a convex function on S, then E[g(X)] \geq g(E[X])

Table of Content