Lecture 011

Ada's Lecture

P vs. NP

$P$: set of languages can be decided in $O(n^k)$ steps for some constant $k$.

$NP$: Might not be in $P$, but verifiable in $P$

• Bounded Entsheidungsproblem: come up with proof that is at most $k$ length long.

• Subset Sum Problem: given a set $S$, find subset $S' \subseteq S$ such that $\sum_S = 0$.

• Traveling Salesperson Problem.

• Satisfiability Problem (SAT): given a boolean formula, determine if it is satisfiable.

• Sudoku Problem: given a partially filled $n \times n$ sudoku board, determine if there is a solution.

Cannot prove these problems are not solvable in polynomial time.

Polynomial Time Reduction: $A \leq^P B \iff M_A \text{ runs } M_B \text{ polynomial number of time to solve problem}$

• Strategy: find some class of hard problems $L$, show that they can be reduced to $A$ to get $L \leq^P A$

$C$-hard: Let $C$ be a set of languages containing $P$. $A$ is $C$-hard if $(\forall L \in C)(L \leq^P A)$ ($A$ is at least as hard as every language in $C$)

• $A$ is outside or equal to the boundary of $C$

$C$-complete: $A$ is $C$-hard and $A \in C$ ($A$ is a representative for hardest language in $C$)

• $A \text{ is } C\text{-hard} \implies (A \in P \iff C e= P)$

• $A$ is equal to the boundary of $C$

Definition of $NP$: Nodeterministic Polynomial Time (it can be determined by nodeterministic TM in polynomial time)

• there is a polynomial time verifier TM $V$

• a polynomial $p(\cdot)$

• for all $x \in \Sigma^*$:

• $x \in L \implies (\exists u)(|u| \leq p(|x|) \implies V(x, u) \text{ accpets})$ (there is poly-length proof that leads to V accept)

• $x \notin L \implies (\forall u)(V(x, u) \text{ rejects})$ (every u leads V to reject)

• or you can say $x \in L \iff V(x, \cdot) \text{ is satisfiable}$

Definition of $EXP$: all languages that can be decided in at most exponential-time (in $O(2^{n^C})$ for $C > 0$)

Facts:

• CIRCUIT-SAT in NP

• P in NP

• NP in EXP

• CIRCUIT-SAT is NP-complete (Cook-Levin Theorem)

• 3SAT is NP-complete

• 3COL is NP-complete

• CLIQUE is NP-complete

• IS is NP-complete

• MSAT (at least 2 different sat) is NP-complete

• BIN is NP-complete

Strategy:

• show L in NP: present a verifier for L

• show L NP-hard: reduce NP-hard or NP-complete problem to L

• show L NP-complete: show L in NP and NP-hard

• if $c$ depends on $n$, then it is not polynomail

• if $c$ is constant, then it is polynomial

• 

• proof of verifier: correctness and time complexity

Properties of $NP$:

• Every decision problem in $NP$ can be solved using brute force search (search space is non-polynomail, but exists verifier)

• The Cook-Levin Theorem: SAT is $NP$-complete ($\forall L \in NP, L \leq^P SAT$)

• Karp's 21 NP-complete problems:

  

• Super Mario Bros, Tetris are NP-complete

Properties of $NP$-complete:

• every $NP$-complete problem reduce to each other

• you can create a Sudoku game to solve conjectures

Reductions

Cook reduction: poly-time Turning reduction ($A \leq^P B$)

• solve $A$ in poly-time using a blackbox oracle that solves B. Only call $M_b$ $poly(|x|)$ times.

• $A \leq^P B \implies (B \in P \implies A \in P)$

• $A \leq^P B \implies (A \notin P \implies B \notin P)$

Karp reduction(poly-time many-one reduction): ($A \leq_m^P B$)

• make ONE call to $M_B$ and directly use its answer as output

• to show $A \leq_m^P B$:

  • define $f : \Sigma^* \rightarrow \Sigma^*$
  • show $x \in A \iff f(x) \in B$
  • show $f$ is poly-time

• 

• Karp reduction is a special case of Cook reduction

Different reduction lead to different notion of $NP$-hardness

Reduction Practices 1

Boolean Circuit: a directed acyclic graph where vertices are AND gates, OR gates, NOT gates, $n$ input gates, $1$ output gate, and constant gates (with reasonable input output numbers). It can be thought as function $f: \{0, 1\}^n \rightarrow \{0, 1\}$.

CIRCUIT-SAT: given circuit, return true iff there exists an assignment of inputs to make it output $1$

kCOL: given $G = \langle{V, E}\rangle$, return true iff $G$ is k-colorable.

CLIQUE: given $\langle{G, k}\rangle$ (graph $G$, positive int $k$), return true iff $G$ contains a clique (all connected, complete graph) of size $k$.

IND-SET: given $\langle{G, k}\rangle$ (graph $G$, positive int $k$), return true iff $G$ contains an independent set (none-connected, no edge between any two vertices in the subset) of size $k$.

Note:

• decision problems is languages

• if input does not correspond to a valid encoding, it is not in the language, input rejected

Show $CLIQUE \leq_m^P IND-SET$: define $f : \overrightarrow{G, k} \rightarrow \langle{G', k'}\rangle$ such that $G$ has a clique of size $k$ iff $G'$ has an independent set of size $k'$ ($f$ is complement of vertices)

Reduction Practice 2

3SAT: give a clause of $A_1 \land A_2 \land A_3...$ where $A_i$ are connected using AND and $A_i = (O_1 \lor O_2 \lor O_3)$ only 3 clause connected using OR. ($\phi = (x_1 \lor \lnot x_2 \lor x_3) \land (\lnot x_1 \lor x_4 \lor x_5) \land (x_2 \lor \lnot x_5 \lor x_6)$) Tell me whether this is satisfiable

Show $3SAT \leq_m^P CLIQUE$:

• 

Sutner's Lecture

NP: Language $L \subseteq \Sigma^*$ is in NP if there is a polynomial time decidable relation $R$ and a polynomial $p$ such that $x \in L \iff \exists w(|w| \leq p(|x|) \land R(w, x))$

VERTEX-COVER: $C \subseteq G$ such that $(\forall (v, u) \in E)(v \in C \lor u \in C)$ and make $C$ small.

• Decision: Does $G$ have a vertex cover of size $k$? (input length for G: $n^2$; k: $\log n$)

• Function: compute the lexicographically first cover of minimal size

• Search: compute a cover of minimal size

CLIQUE: a clique for $G = \langle{V, E}\rangle$ is $C \subseteq V$ such that $(\forall u, v \in C)(u \neq v \implies \{v, u\} \in E)$

• Decision: Does $G$ have a clique of size $k$?

INDEPENDENT-SET: an independent set for $G = \langle{V, E}\rangle$ is $I \subseteq V$ such that $(\forall u, v \in C)(u \neq v \implies \{v, u\} \notin E)$

• Decision: Does $G$ have a independent set of size $k$?

TSP: for $n \times n$ positive integer distance matrix $d$, a bound $B$, is there a tour of cost at most $B$?

• Triangle TSP: the distances are required to be symmetric and conform to triangle inequality $d(i, j) \leq d(i, k) + d(k, j)$

• Euclidean TSP: Points are in the plane with integer coordinates $(x, y)$. Distance defined as Euclidean distance between points.

• Hamiltonian cycle: a path on undirected graph $G$ that contains every vertex of $G$ exactly once. (no known polynomial time solver, related to TSP)

• Eulerian cycle: a path on undirected graph $G$ that contains every vertex of $G$ exactly once and the cycle is created to use every edge of the graph exactly once. (solvable in linear time)

Oracle: assume O(1), used in reasoning of poly-time reduction from non-decision version to decision.

Nodeterministic Polynomial: solvable by NTM in polynomial time.

• however, if NTM $M$ solves problem $P$, then we cannot simply negate the result from $M$ to solve $\lnot P$.

• add unbounded search to $P$ is $NP$

• Formal Definition: $x \in L \iff (\exists w)(|w| \leq p_{\text{olynomial}}(|x|) \land R(w, x))$ where $w$ is certificate (witness, can't be infinitely large otherwise R will never by polynomial and might not halt), $L$ is space (language), $R: W \times X \rightarrow \{0, 1\}$ is verification function, $x$ is input.

• Boolean Operation:

  • Union, Intersection: closed by running two algorithm sequentially

• Complement($co-NP$): $x \notin L \iff (\forall w)(|w| \leq p_{\text{olynomial}}(|x|) \implies \lnot R(w, x))$

  • $P = NP \implies P = NP = co-NP$
  • We don't know if $NP \cap co-NP = P$ hold. (usually work, primality testing is example)

• Intuition Analogy

  • decidable: P
  • semidecidable: NP
  • cosemidecidable: co-NP

Nondeterministic TM: same idea as NFA except

• For non-decision problem: might get different answer in different branch

• Time complexity: $Time_M(x) = \min \{|\beta| : \beta \text{ is accepting branch of } TM_x\}$

• Assumption:

  • $p \rightarrow p_1, p_2, p_3, p_4$ can be treated as $\begin{cases} p \rightarrow q_1, q_2\\ q_1 \rightarrow p_1, p_2\\ q_2 \rightarrow p_3, p_4\\ \end{cases}$ with only $\log n$ slow down
  • so each step will have exactly 2 choices
  • we can attach a clock to terminate all branches when a solution is found
  • We can combine 1 bit at each level and validate solution in regular TM. Therefore we need two deterministic transition function $\begin{cases} \delta_0 : Q \times \Sigma \rightarrow Q \times \Sigma \times \{\pm 1, 0\}\\ \delta_1 : Q \times \Sigma \rightarrow Q \times \Sigma \times \{\pm 1, 0\}\\ \end{cases}$ to make up a non-deterministic transition function. The choice tell us which one to use.

• Space complexity: $Space_M(x) = \min \{|\beta| : \beta \text{ is accepting branch of } TM_x\}$ (notice we ignored time-space trade-off)

Reduction

Polynomial Time Turing Reduction: $\leq^p_T$

• preorder: polynomials are closed under substitution

• $B \in P \land A \leq^P_T \implies A \in P$

• complement: $\bar{A} \leq^P_T A$ is true for all $A$. Therefore it cannot distinguish $co-NP$ from $NP$

• so $B \in NP \land A \leq_T^P \text{ does not imply } A \in NP$ (problem solved by many-one reduction)

Polynomial Time Many-one Reduction: $A \leq^p_m B$ if there is a polynomial time computable function $f$ such that $x \in A \iff f(x) \in B$

• preorder: polynomials are closed under substitution

• $B \in P \land A \leq^p_m B \implies A \in P$

• $B \in NP \land A \leq^p_m B \implies A \in NP$

• NP Hard: $B | (\forall A \in NP)(A \leq^p_m B)$ (lower bound)

• NP-complete: $B | B\in NP \land B\in NP\text{-Hard}$ (lower and upper bound)

• $\exists B \in NP\text{-complete} \land B \in P \implies P = NP$

Universal Machines:

• Enumeration $(M_e)_e$ of all polynomial time Turing machine.

• Universal Machine: $q(n^e + e)$ time ($U(e\# x) = M_e(x)$ is not polynomial because $e$ is variable)

Construct NP-complete: simulate machines in the enumeration $N_e$ of nondeterministic, polynomial time TM.

• $K = \{e \# x | x \text{ accepted by } N_e\}$ is NP-hard, but not in NP (not working)

• 

• 

Boolean Formulae

Boolean Function: formula using $\lnot, \land, \lor$

• input: many boolean

• output: one boolean

For formula $\varphi$, $\varphi[\sigma]$ denotes the truth value of $\varphi$ given $\sigma$.

• satisfiable: $(\exists \sigma)(\varphi [\sigma] = 1)$

• tautology: $(\forall \sigma)(\varphi [\sigma] = 1)$

Threshold function: a boolean function, for $0 \leq m \leq n$, $thr^n_m(x) = \begin{cases} 1 \text{ if } |\{i | x_i = 1\}| \geq m\\ 0 \text{ otherwise}\\ \end{cases}$ where $x_i$ is ith input.

• $thr_0^n = 1$

• $thr_n^1 = \text{n-ary disjunction}$

• $thr_n^n = \text{n-ary conjunction}$

Threshold Function implemented $thr^n_k(x)$ using boolean function: $\bigvee_{I \subset [n] | |I|=k} \bigwedge_{i \in I} x_i$

• the size is $\Theta(k \begin{pmatrix}n\\k\\\end{pmatrix})$

• the size is polynomial in n when k is fixed

Counting Function implemented $cnt^n_k$ using boolean function: $thr^n_k(x) \land \lnot thr^n_{k+1}(x)$

• the size is $\Theta(n^{k+1})$ for fixed $k$

• where the first picture is truth assignment for $cnt^n_1$

• where the last picture is truth assignment for $cnt^n_1$

• therefore SAT looks like NP-complete

• To prove this fact: prove $\Phi_x \text{ is satisfiable } \iff M \text{ accepts } x\#w \text{ for some } w$ the power of a boolean formula is the same as TM solving NP problem (we can simulate NP TM on boolean formula)

Exponential Time Hypothesis: every algorithm for SAT has running time $\Omega(2^{cn})$ for some $c > 0$

3-SAT Formal Definition:

• Literal: a variable or negated variable

• Conjunctive Normal Form (CNF): a conjunction of disunctions of literals ($\Phi_1 \land \Phi_2 \land ... \land \Phi_n$ where $\Phi_i = z_{i, 1} \lor z_{i, 2} \lor ... \lor z_{i, k(i)}$ where $z_{i, j}$ is a literal)

• 3-CNP: $k(i) = 3$ for all $i$

• SAT is NP-complete for formulae in 3-CNF

Cook-Levin

Vertex cover is reducible to 3SAT (proof in slides)

Karp's List

Problems in $P$ that was not known:

• Nonprimes: detect wether a number is not a prime

• Linear Inequalities: whether $Cx \geq d$ has a rational solution

Still unknown complexity: Graph Isomorphism (given two graph $G$ and $G'$, whether $G$ is isomorphic to $G'$)

PARTITION: For $a_1, a_2, ..., a_n \in \mathbb{N}$, is there $I \subset [n]$ such that $\sum_{i \in I} a_i = \sum_{i \notin I} a_i$?

BIN: Can we put $a_1, a_2, ..., a_n \in \mathbb{N}$ in $k \in \mathbb{N}$ bins of size $C \in \mathbb{R}$

Reductions

Examples of Hard reductions: See lecture slide 16.

Numerical Problems

strongly NP-hard: if we express input in unary, it will still remain NP-hard.

• example: TSP, all non-arithmetic NP-hard problems

pseudo-polynomial time: if we express input in unary, it will be in polynomail time.

• example: Partition