# Lecture 000

## Series

f(x) = f(0) + \frac{f'(0)}{1!}x + \frac{f''(0)}{2!}x^2 + \frac{f'''(0)}{3!}x^3 + ...

### Geometric Series

Example 2.1: Evaluate: $S = 1 + x + x^2 + x^3 + ... + x^n$

The idea is to write $S$ in terms of a shorter version of $S$ without using $n$ and the find $S$. (this is equivalent as multiply both side by $(1 - x)$)

\begin{align*} S & = 1 + x + x^2 + x^3 + ... + x^n \\ & = x(S - x^n) + 1 \\ (1 - x)S & = 1 - x^{n + 1} \\ S & = \frac{1 - x^{n + 1}}{1 - x} \\ \end{align*}

Example 2.2: Evaluate: $S = 1 + x + x^2 + x^3 + ...$ for $|x| < 1$ (WARNING: this is infinite series)

The idea is the same, you get $S = \lim_{n \rightarrow \infty} \frac{1 - x^{n + 1}}{1 - x}= \frac{1}{1 - x}$. Be careful, if $|x| \geq 1$, then the series diverge.

Example 2.3: Evaluate $S = 1 + 2x + 3x^2 + 4x^3 + ... + nx^{n - 1}$

This is the derivative of a known sum.

\begin{align*} S & = \frac{d}{dx}(1 + x + x^2 + x^3 + .. + x^n) \\ & = \frac{d}{dx} \frac{1 - x^{n - 1}}{1 - xe} \\ & = \frac{1 - (n + 1)n^x + nx^{n+1}}{(1 - x)^2} \\ \end{align*}

The above assumes $x \neq 1$. When $x = 1$, $S = \frac{n(n + 1)}{2}$

### Taylor Series and Definition of e

\begin{align*} e =& 1 + \frac{1}{1!} + \frac{1}{2!} + \frac{1}{3!} + ... = \lim_{n \rightarrow \infty}(1+\frac{1}{n})^n\\ e^x =& 1 + \frac{x}{1!} + \frac{x^2}{2!} + \frac{x^3}{3!} + ... = \lim_{n \rightarrow \infty} (1 + \frac{x}{n})^n\\ e^x >& 1 + x\\ e^{-1} \leq& 2(1-\frac{1}{n})^2\\ {n \choose k} >& \left(\frac{n}{k}\right)^k\\ \end{align*}

Because

\lim_{n \rightarrow \infty} (1 + \frac{x}{n})^n = \lim_{\frac{n}{x} \rightarrow \infty} (1 + \frac{1}{\frac{n}{x}})^{\frac{nx}{x}} = \lim_{\frac{n}{x} \rightarrow \infty} ((1 + \frac{1}{\frac{n}{x}})^{\frac{n}{x}})^x = e^x

### Harmonic Series

n-th Harmonic Number:

H_n = 1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + ... + \frac{1}{n}

Therefore, we have two inequations:

\left[\begin{cases} H_n >& \int_1^{n+1} \frac{1}{x} dx = \ln(n+1)\\ H_n - \frac{1}{1} <& \int_1^n \frac{1}{x} dx = \ln(n)\\ \end{cases}\right] \implies \begin{align} \ln (n + 1) < H_n &< 1 + \ln (n)\\ H_n &\simeq \ln(n)\\ \lim_{n \rightarrow \infty} H_n &= \infty\\ \end{align}

Similarly

\left[\begin{cases} H_{n, 2} >& \int_1^{n+1} \frac{1}{x^2} dx = \frac{n}{1+n}\\ H_{n, 2} - \frac{1}{1} <& \int_1^n \frac{1}{x^2} dx = \frac{n - 1}{n}\\ \end{cases}\right] \implies \begin{align*} \frac{n}{1+n} < H_{n, 2} &< \frac{2n - 1}{n}\\ H_{n, 2} &\simeq 1\\ \end{align*}

## Differentiation

\begin{align*} \frac{d}{dx} [f(x)]^n =& n[f(x)]^{n - 1}f'(x)\\ \frac{d}{dx} e^{f(x)} =& f'(x)e^{f(x)}\\ \frac{d}{dx} \log_a[f(x)] =& \frac{f'(x)}{\ln a \cdot f(x)}\\ \end{align*}

## Integration

\begin{align*} \int x^n dx =& \frac{1}{n + 1}x^{n + 1} + C\tag{for $n \neq -1$}\\ \int \frac{1}{x} dx =& \ln|x| + C\\ \int \frac{1}{ax + b} dx =& \frac{1}{a} \ln |ax + b| + C\\ \int \ln x dx =& x\ln x - u + C\\ \int ae^{ax} dx =& e^{ax} + C\\ \int f'(x)e^{ax} dx =& e^{f(x)} + C\\ \int a^x dx =& \frac{x^x}{\ln a} + C\\ \int a^x \ln a dx =& a^x + C\\ \end{align*}

Since you know integrate Normal distribution (and any other distribution) gives you $1$. You can modify your equation to match the p.d.f. of a probability to solve your integral.

### Integration by Parts

We have $\int_a^b f'' g dx = |_a^b f' g - \int_a^b f' g' dx$

\left[\begin{align*} & \int_a^b \frac{d}{dx}(uv) \\ = & \int_a^b (v dv + u dv) \\ = & |_a^b uv \\ \end{align*}\right] \implies \int_a^b v dv = [uv]_a^b - \int_a^b u dv

where $du = u'(x) dx$ and $dv = v'(x) dx$

### Change of variable

Change of variable: Change variable dependency in dimensional integration. Please draw a picture.

### Fundamental Theory of Calculus

Fundamental Theory of Calculus:

\frac{d}{dx}\int_a^{g(x)} f(t) dt = f(g(x)) \cdot g'(x)

(The change of the value $\int_a^{g(x)} f(t) dt$ by tweaking $x$ in $g(x)$ is $f(g(x)) \cdot g'(x)$)

## Combinatorics

### Permutation

n(n-1)(n-2)...(n-(k-1)) = \frac{n!}{(n - k)!}

### Combination

C_k^n = {n \choose k} = \frac{n!}{(n - k)! \cdot k!}

### Number of Elements in Power Set

{n \choose 0} + {n \choose 1} + {n \choose 2} + ... + {n \choose n} = 2^n

### Binomial Expansion

{n \choose 0}y^n + {n \choose 1}xy^{n - 1} + {n \choose 2}x^2y^{n - 2} + ... + {n \choose n}x^n = (x + y)^n

${n \choose 0} + {n \choose 1}x + {n \choose 2}x^2+ ... + {n \choose n}x^n = (x + 1)^n$ for special case of binomial expansion when $y = 1$.

### Bounds

Simple bounds on ${n \choose k}$: // TODO: proof

(\frac{n}{k})^k \leq {n \choose k} \leq (\frac{ne}{k})^k

Vandermonde's Identity:

{m + n \choose r} = \sum_{k = 0}^r {m \choose k} \cdot {n \choose r-k}

(choosing $r$ many stuff in mixed $m+n$ population is the same as choosing $k$ in first population and then choosing $r - k$ in second population.)

Stirling Bounds: For all positive integer $n$, (The upper and lower bounds above differ by a multiplicative constant of less than 1.1)

\sqrt{2\pi n}(\frac{n}{e})^n \leq n! \leq e\sqrt{n}(\frac{n}{e})^n

## Asymptotic Notation

$O$: $\leq$ smaller, either significantly or by constant factor

• $f(n) \in O(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} = c$ for some constant $c \geq 0$

$o$: $<$ significantly smaller

• $f(n) \in o(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} = 0$

$\Omega$: $\Omega$ bigger, either significantly or by constant factor

• $f(n) \in \Omega(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} > 0$

$\omega$: $>$ significantly bigger

• $f(n) \in \omega(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} = \infty$

$\Theta$: $=$ is $O$ and $\Omega$

In general:

• $O(g(n))$ is the set of functions that grow no faster than $g(n)$.

• $o(g(n))$ is the set of functions that grow strictly slower than $g(n)$.

• $\Theta(g(n))$ is the set of functions that grow at the same rate as $g(n)$.

• $\Omega(g(n))$ is the set of functions that grow no slower than $g(n)$.

• $\omega(g(n))$ is the set of functions that grow strictly faster than $g(n)$.

Table of Content