Lecture 001

Problems and Coding

Discrete Problems

Types of Problems

Computational problem: $f: \Sigma^* \rightarrow \Sigma^*$

Decision Problems: return true/false

• Example: Primality

  • Instance: a natural number $x$
  • Question: is $x$ a prime number?
  • Definition: select $I_Y \in I \in 2^*$ such that $I_Y \text{ is prime}$ where $I = \mathbb{N}^2$

• Don't cheat by do computation in definition: Encode natural number $n$ as $2n + 1$ whenever $n$ is prime; and as $2n$ when $n$ is compound.

Counting Problems: count, a set of instances $\mathbb{N}$ associate with a set of solutions

• Example: PrimeCounting
  • Instance: A natural number $x$
  • Solution: The number of primes $p \leq x$)
  • Definition: give cardinality of $\{p \in \mathbb{N} \land p \leq x | p \text{ is prime}\}$

Function Problems: calculate a certain function

• Example: Factorization
  • Instance: A natural number $x \geq 2$
  • Solution: A list of the prime factors of $x$.
  • Definition: find the set of solution by $\{sol(x) | x \in \mathbb{N}\}$ (If no solution exists for some instance of a function problem, we will assume that we return an answer No)

Search Problems: select one particular solution

• Example: Factor

  • Instance: A natural number $x \geq 2$
  • Solution: A natural number $z | 1 < z < x$, dividing $x$)
  • Definition: give $\emptyset$ if there isn't, give arbitrary $\{x | x \in \mathbb{N} \land \text{ some criteria}\}$

  Primality are as hard as PrimeCounting Factor and Factorization are closely related

Computability

Computability: computable, decidable, semi-decidable

• General Computability (Classical Theory): natural number only, study of arithmetic function $f: \mathbb{N}^k \rightarrow \mathbb{N}$ and relations $R \subseteq \mathbb{N}^k$

• Concrete Computability: less abstract, other discrite structure like lists

  • no think of data structure
  • all function are arithmetic: $f: \mathbb{N}^k \nrightarrow \mathbb{N}$ (partial)

Complexity: more fined grained computability, related to algorithm

Natural Number: finite von Neumann ordinal

• the leaves are $\emptyset$

• $\{\emptyset\}$ represents 1

• $\{\emptyset, \{\emptyset\}\}$ represents 2

Primitive Recursive Functions

Primitive Recursive Functions (PR): a set of basic functions

• computable: because arithmetic functions that can be defined by recursion is computable.

• forms recursive data type

• closed under certain operations

Components of Primitive Recursion

• Constant Zero: $0$ or $f: \mathbb{N}^0 \rightarrow \mathbb{N}$ given by $f() = 0$ is P.R.

• Successor Function: $S: \mathbb{N} \rightarrow \mathbb{N}, S(x) = x + 1$

• Projections: $P^n_i : \mathbb{N}^n \rightarrow \mathbb{N}, P^n_i(x_1, ..., x_n) = x_i$

• Composition: $f(x) = h(g_1(x), ..., g_n(x)) = \text{Comp}[h, g_1, ..., g_n]$ is primitive recursive when $h(), g()$ are primitive recursive

  • in this case $f: h \cdot (g1, ..., gn)$
  • don't do $add^{(3)}(x, y, z) = add^{(2)}(x, add^{(2)}(y, z))$ because the input type of $x$ does not match type input type of $add^{(2)}(y, z)$
  • do $add^{(3)} = add^{(2)} \circ (P_1^3, add^{(2)} \circ (P_2^3, P_3^3))$

• Primitive Recursion: if $h: \mathbb{N}^{n+2} \rightarrow \mathbb{N}, g: \mathbb{N}^{n} \rightarrow \mathbb{N}$ are P.R, then $f: \mathbb{N}^{n+1} \rightarrow \mathbb{N} = Prec[h, g] = \begin{cases}f(0, y) = g(y)\\f(x+1, y) = h(x, f(x, y), y)\\\end{cases}$ is P.R. where $y$ is the argument being inducted

Example: Factorial

// TODO: 46:02

Relations as Function

Definition: A set (relation) $A \subseteq \mathbb{N}^k$ is primitive recursive if there exists a characteristic function (Decision Problem: $\in$, ie. easy to decide whether an instance is in a particular set) that is primitive recursive.

Characteristic Function:

Primitive Recursive Set: for $A \subseteq \mathbb{N}^k$, $A$ is P.R. if $\text{char}_A$ is P.R.

// EXERCISE: show that these functions are primitive recursive

Lemma: The primitive recursive sets from a Boolean algebra are closed under intersection, union, and complement

• $\text{charFun}_{R \cap S} = \text{mul} \circ (\text{charFun}_{R}, \text{charFun}_{S})$

• $\text{charFun}_{R \cup S} = \text{sign} \circ \text{add} \circ (\text{charFun}_{R}, \text{charFun}_{S})$

• $\text{charFun}_{N-R} = \text{sub} \circ (\text{S} \circ 0, \text{charFun}_{R})$

Any intuitively computable arithmetic function that is not designed maliciously by a logician is primitive recursive.

Arithmetic Coding (handling finite data structure using $\mathbb{N}$)

Problem: find injective, inexpensive function $\langle{.}\rangle : \mathbb{N}^* \rightarrow \mathbb{N}$ (set theory indicate there is bijection, but we need it to be computable)

Pairing System: worry about tuples

• an injective pairing function $\pi : \mathbb{N} \times \mathbb{N} \rightarrow \mathbb{N}$ (because injective, $\pi_1, \pi_2$ always exist)

• The unpairing function $\pi_i : \mathbb{N} \rightarrow \mathbb{N}$ for $i = 1, 2$

Cantor Polynomial

Since $(a, b) \prec (a', b') \equiv \pi(a, b) < \pi(a', b') \equiv (a+b < a'+b') \lor (a+b = a'+b' \land a < b')$, $\pi(x, y) = \frac{(x+2)^2 + 3x + y}{2} = (x^2+y^2+2xy+3x+y)/2$ (Cantor polynomial: a bijection)

// EXERCISE: Explain the Cantor polynomial in terms of traversing the first quadrant and prove that produces a bijection.

// EXERCISE: Find simple descriptions for the corresponding unpairing functions.

// EXERCISE: Show that the unpairing functions are primitive recursive

Even Odd Split

Another pairing function: $\pi(x, y) = 2^x(2y + 1)$ where $(x, y)$ looks like $[y_{\text{binary}}][1][\text{ x many zeros}]$

Coding System

Coding System: consists of coding function, length function, decoding function. The range "Seq" of the coding function is the set of sequence numbers.

(The coding function is not primitive recursive (it is multiadic, coding function takes in a multi-dimensional input, or multiple arguments) because it takes potentially infinite number of arguments. But it does not have to be primitive recursive.)

primitive recursive coding system:

• if $\text{len}$ is P.R.

• if $\text{dec}$ is P.R.

• if the sequence numbers is P.R.

• $\hat{\pi}$ itself is not P.R., but $\forall n \in \mathbb{N}$, $\hat{\pi}_n: \mathbb{N}^n \rightarrow \mathbb{N}$ is P.R.

Sequence Numbers: numbers that are in encoded form

// QUESTION: what do you mean "The range "Seq" of the coding function is the set of sequence numbers."?

for a code $b = \langle{a_1, a_2, ... a_n}\rangle$, $n = \text{len}(b) \land \text{dec}(b, i) = a_i \forall i \in [n]$

Pairing Function for Tuples: $\pi: \mathbb{N}^2 \rightarrow \mathbb{N}$ Pairing Function for Sequence of Length at Least Two: $\hat{\pi}: \mathbb{N}^{\geq 2} \rightarrow \mathbb{N}$

Extending Pairs to Encode More

We restrict input to finite length $n$. Want to know: $\hat{\pi}^{\geq 2} \rightarrow \mathbb{N}$

Observe $40 = \hat{\pi}(4, 4) = \hat{\pi}(4, 1, 1) = \hat{\pi}(4, 1, 0, 1) = \hat{\pi}(4, 1, 0, 0, 1) = ...$, so we to encode length inside encoding. (Intuitively, given a number, we can apply the unpairing function as many time as we want. Each iteration we will get numbers that make sense. But the question is how many time we should apply to get the original numbers? We don't know if we are not given this number.)

Length Encoding: encode the length in it: $<span class="arithmatex"><span class="MathJax_Preview">\langle{.}\rangle:\mathbb{N}^* \rightarrow \mathbb{N} = \begin{cases} \langle{\text{nil}}\rangle := \pi(0, 0)\\ \langle{a}\rangle := \pi(1, a)\\ \langle{a_1, ..., a_n}\rangle := \pi(n, \hat{\pi}(a_1, ..., a_n))\\ \end{cases}</span><script type="math/tex">\langle{.}\rangle:\mathbb{N}^* \rightarrow \mathbb{N} = \begin{cases} \langle{\text{nil}}\rangle := \pi(0, 0)\\ \langle{a}\rangle := \pi(1, a)\\ \langle{a_1, ..., a_n}\rangle := \pi(n, \hat{\pi}(a_1, ..., a_n))\\ \end{cases}$ (now it is injective by induction)

// EXERCISE: Show how to check if a number is a sequence number given dec and len

// EXERCISE: Come up with another reasonable pairing function

// EXERCISE: Suppose $\pi$ is a primitive recursive pairing system. Show that our way of extending $\pi$ to a coding system preserves primitive recursiveness.

String Coding

$\mathbb{N}$: include 0 $\mathbb{N}^*$: all finite-length tuples with elements in $\mathbb{N}$ ($\{(x_1, ..., x_n) : n \in \mathbb{N}, \forall i x_i \in \mathbb{N}\}$)

$\Sigma$: finite non-empty alphabet (symbols, characters) Word (string): (infinite), finite, or empty sequence of alphabet ($\epsilon$: empty word). Note that we usually assume finite length unless otherwise stated. $\Sigma^*$: a (countably infinite) set of all finite words over $\Sigma$ ($\Sigma^* = \{a_1a_2...a_n | n \in \mathbb{N} \land a_i \in \Sigma \text{ for all } i\}$) Language: a finite or countably infinite subset of $L \subseteq \Sigma^*$ over $\Sigma$

• $\{\epsilon\}$ is not the same as $\emptyset$.

Operations on Word $w$

Length: length of a word in $|w|$. If $|w|$ is infinite, then the length is undefined. Reversal: given a word $w$, $w^R$ is the word written in backward. Concatenation: given $u, v \in \Sigma^*$, $uv$ or $u \cdot v$ is concatenation. Power: for $n \in \mathbb{N}$, word $u$, $u^n$ is $u$ concatenating with itself $n$ times. Substring: $u$ is substring of $w$ if $w = xuy$ for some $x, y$.

// TODO: do exercise proof

Operations on Language $L$

Size: $|L|$ either finite or countably infinite Reversal: $L^R = \{w^R \in \Sigma^* | w \in L\}$ (reverse every possible word) Concatenation: $L_1L_2 = L_1 \cdot L_2 = \{uv \in \Sigma^* | u \in L_1 \land v \in L_2\}$ (note $L_1L_2 \neq L_2L_1$) Power: $L^n$ is concatenating $L$ with itself $n$ times. ($L^n = \{u_1u_2\cdot \cdot \cdot u_n \in \Sigma^* | \forall i \in \{1, 2, ..., n\} u_i \in L \}$) Star: for finite language $L$, $L^* = \bigcup_{n \in \mathbb{N}} L^n = \{u_1u_2 \cdot \cdot \cdot u_n \in \Sigma^* | \forall i \in \{1, 2, ..., n\} n \in \mathbb{N}, u_i \in L\} = L \cup LL \cup LLL \cup ...$ (Concatenation to itself countable infinite many times)

// TODO: do multiple exercise

Encoding

Encoding: encoding mapping from a finite or countably infinite set $A$ to $\Sigma$ is $\text{Enc} : A \rightarrow \Sigma^*$. $\langle{a}\rangle$ is the encoded $a \in A$.

• countable sets are encodable

• uncountable sets are not encodable

• $n$ has length $\lfloor\log_k{n}\rfloor + 1$ in base $k$

• encoding function don't have to be injective (natural language are generally not injective)

• a set is countable if its elements are finitely encodable

• a set is uncountable if it $\{0, 1\}^\infty$ can inject to that set

Valid Encoding: If $w \in \Sigma^* \land w = \langle{a}\rangle$, then $w$ is a valid encoding of $A$ Encodable: a set that can be encoded (all finite sets are encodable, some infinite sets are encodable by turing machine, undecidable sets are not encodable)

// TODO: exercise

Computational Problems and Decision Problems

Basically, if we can encode something abstract into a common language (coliqual sense, or basis in linear algebra) written in alphabet $\Sigma$, we can do all the work in that language and translate back.

Computational Problem: $f : \Sigma^* \rightarrow \Sigma^*$ is a computational problem over $\Sigma$

• when $x\in \Sigma^*$ does not necessarily correspond to one instance $I$ (because encoding is not surjective), we need to define an error string $e \in \Sigma^*$ and let $f(x) = e$.

• Converted from $f: I \rightarrow F$ where I is the set of possible input object (instances) and S is the set of possible output objects (solution)

Decision Problem: a function $f : \Sigma^* \rightarrow \{0, 1\}$

• Any word $w \in \Sigma^*$ that does not correspond to an encoding of an instance is mapped to 0 by $f$. (any encoding of instance of non-interest is considered not in the language)

• Language $L$ correspond to decision problem $f : \Sigma^* \rightarrow\{0, 1\}$ iff $\forall w \in L | f(w) = 1$.

Entscheidungs Problem: if there a finitary procedure to determine the validity of a given mathematical statement?

// TODO: check your understanding

Study Tips

intuition: understand concept formal: formal definition examples: ... counter examples: ... results: theorems associated with concepts

Proofs

Add is P.R.

$\text{𝖺𝖽𝖽}_2=\text{𝖯𝗋𝖾𝖼}[h, g]$ where $\begin{cases}g = P_1^1\\h = S \circ P_2^3\\\end{cases}$

Multiply is P.R.

$\begin{cases} \text{mult}_2(0, y) = 0\\ \text{mult}_2(x + 1, y) = y + \text{mult}(x, y)\\ \end{cases}$

Factorial is P.R.

$\begin{cases} \text{fact}(0) = 1\\ \text{fact}(x+1) = (x+1) \cdot \text{fact}(x)\\ \end{cases}$