Lecture 015

Randomized Algorithm: Las Vegas

Random Algorithm

• random bits -> random output, runtime

• we want for all input, expected (or with high probability) runtime is small (this is different than average-case analysis where the input is random)

• robust against adversaries

Deterministic Algorithm

• no random bits

• can be fast on most input

• performance measured by worst case: there can be worst case input

• total runtime decrease probability of error

Las Vegas: always produce correct answer, but with random runtime Monte-Carlo: deterministic time, but answer might be incorrect

Quicksort

Deterministic Quicksort

Deterministic Quicksort: given list $l$

1. pick $l[0]$ as pivot, sort everything with respect to pivot
2. sort left and right of pivot with quicksort

Worst Case of Deterministic is $O(n^2)$

Best Case of Deterministic

If we pick the pivot deterministically, we can always construct a worst-case input.

Random Quicksort

Theorem: for all input $E[C(n)] = O(n \lg n)$

Let us define $X_{ij} = \begin{cases} 1 & \text{if i is compared with j}\\ 0 & \text{otherwise}\\ \end{cases}$

Find k-th Smallest Element

Idea: find the $k$-th smallest item

1. choose a pivot at position $i$
2. sort the list from small to large according to pivot $i$
3. if $k < i$($i$ is the position after sorting), then throw away right side
4. if $k > i$($i$ is the position after sorting), then throw away left side
5. if $k = i$($i$ is the position after sorting), we found it!

Best case bound:

Theorem: the expected number of comparison is bounded by $8n$.

We start using induction. Base case: $E[C(1)] = 1 \leq c \cdot 1$. Assume $E[C(i)] \leq c \cdot i$ for some $c \geq 1$, then:

When $k = (n + 1)/2$ (assuming $n$ is odd), then we have Randomized Median-Select algorithm.

Randomized Algorithm: Monte Carlo

Randomized Matrix-Multiplication Checking

Freivalds' Matrix Multiplication Checking Algorithm: checking $A \cdot B =^? C$.

1. Choose random vector $\vec{r} = \langle{r_1, r_2, ..., r_n}\rangle$ where $r_i \in \{0, 1\}$.
2. if $A(B\vec{r}) \neq C\vec{r}$ return False, else return True

Theorem: for $A \cdot B \neq C$, the error probability is:

If we chose $\vec{r}$ such that $A \cdot B \cdot \vec{r} \neq C \cdot \vec{r}$ when it is really $A \cdot B \neq C$, then we say $\vec{r}$ witness the fact $A \cdot B \neq C$.

Proof: given $D := AB - C \neq 0$, what is the probability $D\vec{r} = AB\vec{r} - C\vec{r} = \vec{0}$? Well, since $D \neq 0$, let $d_{1, 1}$ be the non-zero entry of $D$.

If we want to pass the test by maliciously choosing $r$, $r_1$ is determined when $r_2, r_3, ..., r_n$ is determined. Therefore, for every set of $r_2, r_3, ..., r_n$, there is one $r_1$ that makes the malicious injection successful. Since $r_1 \in \{0, 1\}$, the probability such that our choice $r_1$ magically equal to $-\frac{1}{d_{1, 1}} \sum_{j = 2}^n (d_{1, j} r_j)$ is $\leq \frac{1}{|\{0, 1\}|}$.

Note the $\leq$ is used above. We could have the case that none of our choice will make us pass the test due to the chance that $-\frac{1}{d_{1, 1}} \sum_{j = 2}^n (d_{1, j} r_j) \not\in \{0, 1\}$.

The algorithm is $\in \Theta(n^2)$. We can boost it by running $k$ times with complexity $\in \Theta(kn^2)$ to achieve error rate $\leq \left(\frac{1}{|\{0, 1\}|}\right)^k$. We can also boost it by choosing $\vec{r} \in \{0, 1, 2\}^n$ rather than $\vec{r} \in \{0, 1\}^n$.

What is some of our choice of random vector $\vec{r}$ repeat in our process of boosting? This turns out is not a problem as long as we choose vectors independently. What if we have an algorithm with 2-sided error? Then we can do majority voting.

Randomized Polynomial Checking

We would like to know if a given polynomial $G(x) := x^3 - x^2 - 41x +105$ is equal to $G(X) := (x - 3)(x - 5)(x + 7)$ without simplifying them (as it is expensive $\in O(d^2)$).

Let $d$ be the number of root (degree) of polynomial.

Simple Random Checker:

1. pick a random value $r$ out of $n \cdot d$ many possible choices
2. if $F(r) = G(r)$ return True, otherwise return False.

The above algorithm is $\in \Theta(d)$

Proof: Let $H(x) = F(x) - G(x)$, observe $H(x)$ has at most $d$ roots.

Given $F(x) \neq G(x)$:

Note the $\leq$ is used above. We could have the case that none of our choice are roots to $H(x)$.

Again, we can boost it to achieve error rate $\leq \frac{1}{n^k}$ by running it $k$ times with complexity $\Theta(kd)$.

Randomized Min-Cut

Cut-set: a set of edges whose removal will break the graph into two or more connected components.

The algorithm: given graph $G = \langle{E, V}\rangle$. Let $|V| = n$, $C$ be the set of edges represent one of those min-cuts and $|C| = k$.

1. For each iteration, contract two vertice by selecting a random common edge $e \in E$ between $v_1, v_2$.
2. Stop until there is only two vertice left

Since there is $n$ vertice, we need $n - 2$ iterations. We know the graph has at least $\frac{nk}{2}$ edges by edge counting since $k$ is the length of smallest cut set.

Let $E_i$ denotes the event "no edge of $C$ is selected in the $i$-th round". We calculate $E_1$:

Therefore the probability that the algorithm doesn't output a specific min-cut is: $\leq 1 - \frac{2}{n(n - 1)}$. It is less than a high probability. (It is unlikely it will produce min-cut)

To fix it, we run $\Theta(n^2\ln n)$ times: