Lecture 000
Series
f(x) = f(0) + \frac{f'(0)}{1!}x + \frac{f''(0)}{2!}x^2 + \frac{f'''(0)}{3!}x^3 + ...
Geometric Series
Example 2.1 : Evaluate: S = 1 + x + x^2 + x^3 + ... + x^n
The idea is to write S in terms of a shorter version of S without using n and the find S . (this is equivalent as multiply both side by (1 - x) )
\begin{align*}
S & = 1 + x + x^2 + x^3 + ... + x^n \\
& = x(S - x^n) + 1 \\
(1 - x)S & = 1 - x^{n + 1} \\
S & = \frac{1 - x^{n + 1}}{1 - x} \\
\end{align*}
Example 2.2 : Evaluate: S = 1 + x + x^2 + x^3 + ... for |x| < 1 (WARNING: this is infinite series)
The idea is the same, you get S = \lim_{n \rightarrow \infty} \frac{1 - x^{n + 1}}{1 - x}= \frac{1}{1 - x} . Be careful, if |x| \geq 1 , then the series diverge.
Example 2.3 : Evaluate S = 1 + 2x + 3x^2 + 4x^3 + ... + nx^{n - 1}
This is the derivative of a known sum.
\begin{align*}
S & = \frac{d}{dx}(1 + x + x^2 + x^3 + .. + x^n) \\
& = \frac{d}{dx} \frac{1 - x^{n - 1}}{1 - xe} \\
& = \frac{1 - (n + 1)n^x + nx^{n+1}}{(1 - x)^2} \\
\end{align*}
The above assumes x \neq 1 . When x = 1 , S = \frac{n(n + 1)}{2}
Taylor Series and Definition of e
\begin{align*}
e =& 1 + \frac{1}{1!} + \frac{1}{2!} + \frac{1}{3!} + ... = \lim_{n \rightarrow \infty}(1+\frac{1}{n})^n\\
e^x =& 1 + \frac{x}{1!} + \frac{x^2}{2!} + \frac{x^3}{3!} + ... = \lim_{n \rightarrow \infty} (1 + \frac{x}{n})^n\\
e^x >& 1 + x\\
e^{-1} \leq& 2(1-\frac{1}{n})^2\\
{n \choose k} >& \left(\frac{n}{k}\right)^k\\
\end{align*}
Because
\lim_{n \rightarrow \infty} (1 + \frac{x}{n})^n = \lim_{\frac{n}{x} \rightarrow \infty} (1 + \frac{1}{\frac{n}{x}})^{\frac{nx}{x}} = \lim_{\frac{n}{x} \rightarrow \infty} ((1 + \frac{1}{\frac{n}{x}})^{\frac{n}{x}})^x = e^x
Harmonic Series
n-th Harmonic Number:
H_n = 1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + ... + \frac{1}{n}
H_n and \frac{1}{x} .
Therefore, we have two inequations:
\left[\begin{cases}
H_n >& \int_1^{n+1} \frac{1}{x} dx = \ln(n+1)\\
H_n - \frac{1}{1} <& \int_1^n \frac{1}{x} dx = \ln(n)\\
\end{cases}\right] \implies
\begin{align}
\ln (n + 1) < H_n &< 1 + \ln (n)\\
H_n &\simeq \ln(n)\\
\lim_{n \rightarrow \infty} H_n &= \infty\\
\end{align}
Similarly
\left[\begin{cases}
H_{n, 2} >& \int_1^{n+1} \frac{1}{x^2} dx = \frac{n}{1+n}\\
H_{n, 2} - \frac{1}{1} <& \int_1^n \frac{1}{x^2} dx = \frac{n - 1}{n}\\
\end{cases}\right] \implies
\begin{align*}
\frac{n}{1+n} < H_{n, 2} &< \frac{2n - 1}{n}\\
H_{n, 2} &\simeq 1\\
\end{align*}
Differentiation
\begin{align*}
\frac{d}{dx} [f(x)]^n =& n[f(x)]^{n - 1}f'(x)\\
\frac{d}{dx} e^{f(x)} =& f'(x)e^{f(x)}\\
\frac{d}{dx} \log_a[f(x)] =& \frac{f'(x)}{\ln a \cdot f(x)}\\
\end{align*}
Integration
\begin{align*}
\int x^n dx =& \frac{1}{n + 1}x^{n + 1} + C\tag{for $n \neq -1$}\\
\int \frac{1}{x} dx =& \ln|x| + C\\
\int \frac{1}{ax + b} dx =& \frac{1}{a} \ln |ax + b| + C\\
\int \ln x dx =& x\ln x - u + C\\
\int ae^{ax} dx =& e^{ax} + C\\
\int f'(x)e^{ax} dx =& e^{f(x)} + C\\
\int a^x dx =& \frac{x^x}{\ln a} + C\\
\int a^x \ln a dx =& a^x + C\\
\end{align*}
Since you know integrate Normal distribution (and any other distribution) gives you 1 . You can modify your equation to match the p.d.f. of a probability to solve your integral.
Integration by Parts
Integration by parts
We have \int_a^b f'' g dx = |_a^b f' g - \int_a^b f' g' dx
\left[\begin{align*}
& \int_a^b \frac{d}{dx}(uv) \\
= & \int_a^b (v dv + u dv) \\
= & |_a^b uv \\
\end{align*}\right]
\implies
\int_a^b v dv = [uv]_a^b - \int_a^b u dv
where du = u'(x) dx and dv = v'(x) dx
Change of variable
Change of variable: Change variable dependency in dimensional integration. Please draw a picture.
Fundamental Theory of Calculus
Fundamental Theory of Calculus:
\frac{d}{dx}\int_a^{g(x)} f(t) dt = f(g(x)) \cdot g'(x)
(The change of the value \int_a^{g(x)} f(t) dt by tweaking x in g(x) is f(g(x)) \cdot g'(x) )
Combinatorics
Permutation
n(n-1)(n-2)...(n-(k-1)) = \frac{n!}{(n - k)!}
Combination
C_k^n = {n \choose k} = \frac{n!}{(n - k)! \cdot k!}
Number of Elements in Power Set
{n \choose 0} + {n \choose 1} + {n \choose 2} + ... + {n \choose n} = 2^n
Binomial Expansion
{n \choose 0}y^n + {n \choose 1}xy^{n - 1} + {n \choose 2}x^2y^{n - 2} + ... + {n \choose n}x^n = (x + y)^n
{n \choose 0} + {n \choose 1}x + {n \choose 2}x^2+ ... + {n \choose n}x^n = (x + 1)^n for special case of binomial expansion when y = 1 .
Bounds
Simple bounds on {n \choose k} : // TODO: proof
(\frac{n}{k})^k \leq {n \choose k} \leq (\frac{ne}{k})^k
Vandermonde's Identity :
{m + n \choose r} = \sum_{k = 0}^r {m \choose k} \cdot {n \choose r-k}
(choosing r many stuff in mixed m+n population is the same as choosing k in first population and then choosing r - k in second population.)
Stirling Bounds : For all positive integer n , (The upper and lower bounds above differ by a multiplicative constant of less than 1.1)
\sqrt{2\pi n}(\frac{n}{e})^n \leq n! \leq e\sqrt{n}(\frac{n}{e})^n
Asymptotic Notation
O : \leq smaller, either significantly or by constant factor
f(n) \in O(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} = c for some constant c \geq 0
o : < significantly smaller
f(n) \in o(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} = 0
\Omega : \Omega bigger, either significantly or by constant factor
f(n) \in \Omega(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} > 0
\omega : > significantly bigger
f(n) \in \omega(g(n)) \iff \lim_{n \rightarrow \infty} \frac{f(n)}{g(n)} = \infty
\Theta : = is O and \Omega
In general:
O(g(n)) is the set of functions that grow no faster than g(n) .
o(g(n)) is the set of functions that grow strictly slower than g(n) .
\Theta(g(n)) is the set of functions that grow at the same rate as g(n) .
\Omega(g(n)) is the set of functions that grow no slower than g(n) .
\omega(g(n)) is the set of functions that grow strictly faster than g(n) .
Practice Questions