Continuous Random Variable: uncountable set of possible values.
probability density function: a non-negative function such that
Note: because it is continuous f_X(x) dx \simeq Pr\{x \leq X \leq x + dx\}, Pr\{a \leq X \leq b\} = Pr\{a < X < b\} hold
cumulative distribution function:
Note: cdf is strongly tight to pdf f_X(x) = \frac{d}{dx}\int_{-\infty}^x f_X(t) dt = \frac{d}{dx} F_X(x) by Fundamental Theorem of Calculus
tail:
For X \sim \text{U}(a, b):
Canonical Random Variable: Since Uniform [0, 1) is used a lot, we denote \xi = \text{Uniform}(0, 1)
Exponential: probability density function drops off exponentially. X \sim \text{Exp}(\lambda) where \lambda > 0 is the rate of exponential. \lambda is reciprocal of the expectation.
Both f_X(x), \bar{F}_X(x) drop off by a constant factor e^{-\lambda} with each unit increase of x.
The minimum of two exponential is: f_{\min(\text{Exponential}(a), \text{Exponential}(b))} = f_{\text{Exponential}(a+b)} (proved by c.d.f.), and the maximum of two exponential is first occur plus the probability second occur after first (start over again by memoryless)
Bad Example: A person travel from point A to point B with distance d. The speed S \sim \text{U}(\alpha, \beta)chosen on the start. Let T denotes the distribution of time spend. What is E[T].
IDEA 1: E[T] = E[\frac{d}{S}] = \frac{d}{E[S]}. WRONG: although T is uniform, \frac{d}{S} is not for constant d. It is similar to f(x) = \frac{1}{x}. Input E[S] cannot bisect T into equal portion. (In fact, E[\frac{d}{S}] = E[d] \cdot E[\frac{1}{S}] = dE[\frac{1}{S}] by constants are trivially independent)
IDEA 2: E[T] = \text{avg}(\frac{d}{\alpha}, \frac{d}{\beta}). WRONG: although T is uniform, \frac{d}{S} is not for constant d. It is similar to f(x) = \frac{1}{x}.
IDEA 3: E[T] = \int_\alpha^\beta \frac{d}{s} \cdot f_S(s) ds. CORRECT: by definition.
There are differences with discrete R.V:
Law of Total Probability for Continuous R.V.s: for event A
f_X(x \cap A) represent the new density function for both X = x and A to happen.
we see Pr\{A | X = x\} = \frac{Pr\{A \cap Pr\{X = x\}\}}{Pr\{X = x\}} = \frac{f_X(x \cap A)}{f_X(x)} make sense because we have both zero in nominator and denominator.
Bayes Law: for continuous random variable X and event A. The conditional density function of X given A is:
Observe \int_x f_{X | A}(x) = 1
Conditional Expectation:
Example: what is the expected probability P \sim \text{U}(0, 1) of head of a unknown biased coin given we have an experiment where we had 10 heads out of 10 flips.
WRONG: our intuition gives E[P|A] = 1. Although it is most likely that the probability of the coin is 1 (meaning \max(f_P(x)) occur at x = 1), but we are asking for the expectation rather than maximum probability.
CORRECT: Let A denotes the event, then E[P|A] = \int_0^1 f_{P | A}(p) \cdot p dp gives us \frac{11}{12}.
Joint Distribution: for continuous random variable X, Y
where \int_{-\infty}^\infty \int_{-\infty}^\infty f_{X, Y}(x, y) dx dy = 1
Marginal densities:
Note that f_X(x), f_Y(y) are densities, not probability. This means although it has the same unit, it is the probability when the domain interval is 1. Be careful! For descrite variables, the probability is the area in p.d.f. graph, which sum to one. For continuous variables, the density is the height in p.d.f. graph, which integrates to one, but does not sum to infinity.
Independence: X \perp Y \iff f_{X, Y}(x, y) = f_X(x) \cdot f_Y(y)
Functional Expectation:
Bayes Law for Multiple Random Variables:
Corollary:
E[X | Y = y] = \int_x x \cdot f_{X | Y = y}(x) dx
E[X] = \int_y E[X | Y = y] \cdot f_Y(y) dy
Table of Content