P: set of languages can be decided in O(n^k) steps for some constant k.
NP: Might not be in P, but verifiable in P
Bounded Entsheidungsproblem: come up with proof that is at most k length long.
Subset Sum Problem: given a set S, find subset S' \subseteq S such that \sum_S = 0.
Traveling Salesperson Problem.
Satisfiability Problem (SAT): given a boolean formula, determine if it is satisfiable.
Sudoku Problem: given a partially filled n \times n sudoku board, determine if there is a solution.
Cannot prove these problems are not solvable in polynomial time.
Polynomial Time Reduction: A \leq^P B \iff M_A \text{ runs } M_B \text{ polynomial number of time to solve problem}
C-hard: Let C be a set of languages containing P. A is C-hard if (\forall L \in C)(L \leq^P A) (A is at least as hard as every language in C)
C-complete: A is C-hard and A \in C (A is a representative for hardest language in C)
A \text{ is } C\text{-hard} \implies (A \in P \iff C e= P)
A is equal to the boundary of C
Definition of NP: Nodeterministic Polynomial Time (it can be determined by nodeterministic TM in polynomial time)
there is a polynomial time verifier TM V
a polynomial p(\cdot)
for all x \in \Sigma^*:
x \in L \implies (\exists u)(|u| \leq p(|x|) \implies V(x, u) \text{ accpets}) (there is poly-length proof that leads to V accept)
x \notin L \implies (\forall u)(V(x, u) \text{ rejects}) (every u leads V to reject)
or you can say x \in L \iff V(x, \cdot) \text{ is satisfiable}
Definition of EXP: all languages that can be decided in at most exponential-time (in O(2^{n^C}) for C > 0)
Facts:
CIRCUIT-SAT in NP
P in NP
NP in EXP
CIRCUIT-SAT is NP-complete (Cook-Levin Theorem)
3SAT is NP-complete
3COL is NP-complete
CLIQUE is NP-complete
IS is NP-complete
MSAT (at least 2 different sat) is NP-complete
BIN is NP-complete
Strategy:
show L in NP: present a verifier for L
show L NP-hard: reduce NP-hard or NP-complete problem to L
show L NP-complete: show L in NP and NP-hard
if c depends on n, then it is not polynomail
if c is constant, then it is polynomial
proof of verifier: correctness and time complexity
Properties of NP:
Every decision problem in NP can be solved using brute force search (search space is non-polynomail, but exists verifier)
The Cook-Levin Theorem: SAT is NP-complete (\forall L \in NP, L \leq^P SAT)
Karp's 21 NP-complete problems:
Super Mario Bros, Tetris are NP-complete
Properties of NP-complete:
every NP-complete problem reduce to each other
you can create a Sudoku game to solve conjectures
Cook reduction: poly-time Turning reduction (A \leq^P B)
solve A in poly-time using a blackbox oracle that solves B. Only call M_b poly(|x|) times.
A \leq^P B \implies (B \in P \implies A \in P)
A \leq^P B \implies (A \notin P \implies B \notin P)
Karp reduction(poly-time many-one reduction): (A \leq_m^P B)
make ONE call to M_B and directly use its answer as output
to show A \leq_m^P B:
Karp reduction is a special case of Cook reduction
Different reduction lead to different notion of NP-hardness
Boolean Circuit: a directed acyclic graph where vertices are AND gates, OR gates, NOT gates, n input gates, 1 output gate, and constant gates (with reasonable input output numbers). It can be thought as function f: \{0, 1\}^n \rightarrow \{0, 1\}.
CIRCUIT-SAT: given circuit, return true iff there exists an assignment of inputs to make it output 1
kCOL: given G = \langle{V, E}\rangle, return true iff G is k-colorable.
CLIQUE: given \langle{G, k}\rangle (graph G, positive int k), return true iff G contains a clique (all connected, complete graph) of size k.
IND-SET: given \langle{G, k}\rangle (graph G, positive int k), return true iff G contains an independent set (none-connected, no edge between any two vertices in the subset) of size k.
Note:
decision problems is languages
if input does not correspond to a valid encoding, it is not in the language, input rejected
Show CLIQUE \leq_m^P IND-SET: define f : \overrightarrow{G, k} \rightarrow \langle{G', k'}\rangle such that G has a clique of size k iff G' has an independent set of size k' (f is complement of vertices)
3SAT: give a clause of A_1 \land A_2 \land A_3... where A_i are connected using AND and A_i = (O_1 \lor O_2 \lor O_3) only 3 clause connected using OR. (\phi = (x_1 \lor \lnot x_2 \lor x_3) \land (\lnot x_1 \lor x_4 \lor x_5) \land (x_2 \lor \lnot x_5 \lor x_6)) Tell me whether this is satisfiable
Show 3SAT \leq_m^P CLIQUE:
NP: Language L \subseteq \Sigma^* is in NP if there is a polynomial time decidable relation R and a polynomial p such that x \in L \iff \exists w(|w| \leq p(|x|) \land R(w, x))
VERTEX-COVER: C \subseteq G such that (\forall (v, u) \in E)(v \in C \lor u \in C) and make C small.
Decision: Does G have a vertex cover of size k? (input length for G: n^2; k: \log n)
Function: compute the lexicographically first cover of minimal size
Search: compute a cover of minimal size
CLIQUE: a clique for G = \langle{V, E}\rangle is C \subseteq V such that (\forall u, v \in C)(u \neq v \implies \{v, u\} \in E)
INDEPENDENT-SET: an independent set for G = \langle{V, E}\rangle is I \subseteq V such that (\forall u, v \in C)(u \neq v \implies \{v, u\} \notin E)
TSP: for n \times n positive integer distance matrix d, a bound B, is there a tour of cost at most B?
Triangle TSP: the distances are required to be symmetric and conform to triangle inequality d(i, j) \leq d(i, k) + d(k, j)
Euclidean TSP: Points are in the plane with integer coordinates (x, y). Distance defined as Euclidean distance between points.
Hamiltonian cycle: a path on undirected graph G that contains every vertex of G exactly once. (no known polynomial time solver, related to TSP)
Eulerian cycle: a path on undirected graph G that contains every vertex of G exactly once and the cycle is created to use every edge of the graph exactly once. (solvable in linear time)
Oracle: assume O(1), used in reasoning of poly-time reduction from non-decision version to decision.
Nodeterministic Polynomial: solvable by NTM in polynomial time.
however, if NTM M solves problem P, then we cannot simply negate the result from M to solve \lnot P.
add unbounded search to P is NP
Formal Definition: x \in L \iff (\exists w)(|w| \leq p_{\text{olynomial}}(|x|) \land R(w, x)) where w is certificate (witness, can't be infinitely large otherwise R will never by polynomial and might not halt), L is space (language), R: W \times X \rightarrow \{0, 1\} is verification function, x is input.
Boolean Operation:
Complement(co-NP): x \notin L \iff (\forall w)(|w| \leq p_{\text{olynomial}}(|x|) \implies \lnot R(w, x))
Intuition Analogy
Nondeterministic TM: same idea as NFA except
For non-decision problem: might get different answer in different branch
Time complexity: Time_M(x) = \min \{|\beta| : \beta \text{ is accepting branch of } TM_x\}
Assumption:
Space complexity: Space_M(x) = \min \{|\beta| : \beta \text{ is accepting branch of } TM_x\} (notice we ignored time-space trade-off)
Polynomial Time Turing Reduction: \leq^p_T
preorder: polynomials are closed under substitution
B \in P \land A \leq^P_T \implies A \in P
complement: \bar{A} \leq^P_T A is true for all A. Therefore it cannot distinguish co-NP from NP
so B \in NP \land A \leq_T^P \text{ does not imply } A \in NP (problem solved by many-one reduction)
Polynomial Time Many-one Reduction: A \leq^p_m B if there is a polynomial time computable function f such that x \in A \iff f(x) \in B
preorder: polynomials are closed under substitution
B \in P \land A \leq^p_m B \implies A \in P
B \in NP \land A \leq^p_m B \implies A \in NP
NP Hard: B | (\forall A \in NP)(A \leq^p_m B) (lower bound)
NP-complete: B | B\in NP \land B\in NP\text{-Hard} (lower and upper bound)
\exists B \in NP\text{-complete} \land B \in P \implies P = NP
Universal Machines:
Enumeration (M_e)_e of all polynomial time Turing machine.
Universal Machine: q(n^e + e) time (U(e\# x) = M_e(x) is not polynomial because e is variable)
Construct NP-complete: simulate machines in the enumeration N_e of nondeterministic, polynomial time TM.
K = \{e \# x | x \text{ accepted by } N_e\} is NP-hard, but not in NP (not working)
Boolean Function: formula using \lnot, \land, \lor
input: many boolean
output: one boolean
For formula \varphi, \varphi[\sigma] denotes the truth value of \varphi given \sigma.
satisfiable: (\exists \sigma)(\varphi [\sigma] = 1)
tautology: (\forall \sigma)(\varphi [\sigma] = 1)
Threshold function: a boolean function, for 0 \leq m \leq n, thr^n_m(x) = \begin{cases} 1 \text{ if } |\{i | x_i = 1\}| \geq m\\ 0 \text{ otherwise}\\ \end{cases} where x_i is ith input.
thr_0^n = 1
thr_n^1 = \text{n-ary disjunction}
thr_n^n = \text{n-ary conjunction}
Threshold Function implemented thr^n_k(x) using boolean function: \bigvee_{I \subset [n] | |I|=k} \bigwedge_{i \in I} x_i
the size is \Theta(k \begin{pmatrix}n\\k\\\end{pmatrix})
the size is polynomial in n when k is fixed
Counting Function implemented cnt^n_k using boolean function: thr^n_k(x) \land \lnot thr^n_{k+1}(x)
where the first picture is truth assignment for cnt^n_1
where the last picture is truth assignment for cnt^n_1
therefore SAT looks like NP-complete
To prove this fact: prove \Phi_x \text{ is satisfiable } \iff M \text{ accepts } x\#w \text{ for some } w the power of a boolean formula is the same as TM solving NP problem (we can simulate NP TM on boolean formula)
Exponential Time Hypothesis: every algorithm for SAT has running time \Omega(2^{cn}) for some c > 0
3-SAT Formal Definition:
Literal: a variable or negated variable
Conjunctive Normal Form (CNF): a conjunction of disunctions of literals (\Phi_1 \land \Phi_2 \land ... \land \Phi_n where \Phi_i = z_{i, 1} \lor z_{i, 2} \lor ... \lor z_{i, k(i)} where z_{i, j} is a literal)
3-CNP: k(i) = 3 for all i
SAT is NP-complete for formulae in 3-CNF
Vertex cover is reducible to 3SAT (proof in slides)
Problems in P that was not known:
Nonprimes: detect wether a number is not a prime
Linear Inequalities: whether Cx \geq d has a rational solution
Still unknown complexity: Graph Isomorphism (given two graph G and G', whether G is isomorphic to G')
PARTITION: For a_1, a_2, ..., a_n \in \mathbb{N}, is there I \subset [n] such that \sum_{i \in I} a_i = \sum_{i \notin I} a_i?
BIN: Can we put a_1, a_2, ..., a_n \in \mathbb{N} in k \in \mathbb{N} bins of size C \in \mathbb{R}
Examples of Hard reductions: See lecture slide 16.
strongly NP-hard: if we express input in unary, it will still remain NP-hard.
pseudo-polynomial time: if we express input in unary, it will be in polynomail time.
Table of Content