Lecture 001

Problems and Coding

Discrete Problems

Types of Problems

Computational problem: f: \Sigma^* \rightarrow \Sigma^*

Decision Problems: return true/false

Counting Problems: count, a set of instances \mathbb{N} associate with a set of solutions

Function Problems: calculate a certain function

Search Problems: select one particular solution


Computability: computable, decidable, semi-decidable

Complexity: more fined grained computability, related to algorithm

Natural Number: finite von Neumann ordinal

finite von Neumann ordinal, representing 5

finite von Neumann ordinal, representing 5

Primitive Recursive Functions

Primitive Recursive Functions (PR): a set of basic functions

Components of Primitive Recursion

Example: Factorial

// TODO: 46:02

Relations as Function

Definition: A set (relation) A \subseteq \mathbb{N}^k is primitive recursive if there exists a characteristic function (Decision Problem: \in, ie. easy to decide whether an instance is in a particular set) that is primitive recursive.

Characteristic Function:

\text{char}_A : \mathbb{N}^k \rightarrow \mathbb{N} = \text{char}_A(x) = \begin{cases} 1 \text{ if } x \in A\\0 \text{ otherwise} \end{cases}

Primitive Recursive Set: for A \subseteq \mathbb{N}^k, A is P.R. if \text{char}_A is P.R.

Examples of primitive recursive

Examples of primitive recursive
// EXERCISE: show that these functions are primitive recursive

Lemma: The primitive recursive sets from a Boolean algebra are closed under intersection, union, and complement

Any intuitively computable arithmetic function that is not designed maliciously by a logician is primitive recursive.

Arithmetic Coding (handling finite data structure using \mathbb{N})

Problem: find injective, inexpensive function \langle{.}\rangle : \mathbb{N}^* \rightarrow \mathbb{N} (set theory indicate there is bijection, but we need it to be computable)

Pairing System: worry about tuples

Cantor Polynomial

Since (a, b) \prec (a', b') \equiv \pi(a, b) < \pi(a', b') \equiv (a+b < a'+b') \lor (a+b = a'+b' \land a < b'), \pi(x, y) = \frac{(x+2)^2 + 3x + y}{2} = (x^2+y^2+2xy+3x+y)/2 (Cantor polynomial: a bijection)

Cantor polynomial

Cantor polynomial

// EXERCISE: Explain the Cantor polynomial in terms of traversing the first quadrant and prove that produces a bijection.

// EXERCISE: Find simple descriptions for the corresponding unpairing functions.

// EXERCISE: Show that the unpairing functions are primitive recursive

Even Odd Split

Another pairing function: \pi(x, y) = 2^x(2y + 1) where (x, y) looks like [y_{\text{binary}}][1][\text{ x many zeros}]

Coding System

Coding System: consists of coding function, length function, decoding function. The range "Seq" of the coding function is the set of sequence numbers.

\begin{cases} \langle{.}\rangle : \mathbb{N}^* \rightarrow \mathbb{N}\\ \text{len}:\mathbb{N} \rightarrow \mathbb{N}\\ \text{dec}:\mathbb{N} \times \mathbb{N} \rightarrow \mathbb{N}\\ \end{cases}
(\forall (a_1, ..., a_n) \in \mathbb{N}^*) (\forall i \in \{1, ..., n\}) \begin{cases} \text{len}(\langle{a_1, ..., a_n}\rangle) = n\\ \text{dec}(\langle{a_1, ..., a_n}\rangle, i) = a_i\\ \end{cases}

(The coding function is not primitive recursive (it is multiadic, coding function takes in a multi-dimensional input, or multiple arguments) because it takes potentially infinite number of arguments. But it does not have to be primitive recursive.)

primitive recursive coding system:

Sequence Numbers: numbers that are in encoded form

// QUESTION: what do you mean "The range "Seq" of the coding function is the set of sequence numbers."?

for a code b = \langle{a_1, a_2, ... a_n}\rangle, n = \text{len}(b) \land \text{dec}(b, i) = a_i \forall i \in [n]

Pairing Function for Tuples: \pi: \mathbb{N}^2 \rightarrow \mathbb{N} Pairing Function for Sequence of Length at Least Two: \hat{\pi}: \mathbb{N}^{\geq 2} \rightarrow \mathbb{N}

Extending Pairs to Encode More

We restrict input to finite length n. Want to know: \hat{\pi}^{\geq 2} \rightarrow \mathbb{N}

\begin{cases} \hat{\pi}(x_1, x_2) = \pi(x_1, x_2)\\ \hat{\pi}(x_1, x_2, ..., x_k) = \pi(x_1, \hat{\pi}(x_2, ..., x_k))\\ \end{cases}

Left, Right associative

Left, Right associative

Observe 40 = \hat{\pi}(4, 4) = \hat{\pi}(4, 1, 1) = \hat{\pi}(4, 1, 0, 1) = \hat{\pi}(4, 1, 0, 0, 1) = ..., so we to encode length inside encoding. (Intuitively, given a number, we can apply the unpairing function as many time as we want. Each iteration we will get numbers that make sense. But the question is how many time we should apply to get the original numbers? We don't know if we are not given this number.)

Length Encoding: encode the length in it: \langle{.}\rangle:\mathbb{N}^* \rightarrow \mathbb{N} = \begin{cases} \langle{\text{nil}}\rangle := \pi(0, 0)\\ \langle{a}\rangle := \pi(1, a)\\ \langle{a_1, ..., a_n}\rangle := \pi(n, \hat{\pi}(a_1, ..., a_n))\\ \end{cases} (now it is injective by induction)

// EXERCISE: Show how to check if a number is a sequence number given dec and len

// EXERCISE: Come up with another reasonable pairing function

// EXERCISE: Suppose \pi is a primitive recursive pairing system. Show that our way of extending \pi to a coding system preserves primitive recursiveness.

String Coding

\mathbb{N}: include 0 \mathbb{N}^*: all finite-length tuples with elements in \mathbb{N} (\{(x_1, ..., x_n) : n \in \mathbb{N}, \forall i x_i \in \mathbb{N}\})

\Sigma: finite non-empty alphabet (symbols, characters) Word (string): (infinite), finite, or empty sequence of alphabet (\epsilon: empty word). Note that we usually assume finite length unless otherwise stated. \Sigma^*: a (countably infinite) set of all finite words over \Sigma (\Sigma^* = \{a_1a_2...a_n | n \in \mathbb{N} \land a_i \in \Sigma \text{ for all } i\}) Language: a finite or countably infinite subset of L \subseteq \Sigma^* over \Sigma

Operations on Word w

Length: length of a word in |w|. If |w| is infinite, then the length is undefined. Reversal: given a word w, w^R is the word written in backward. Concatenation: given u, v \in \Sigma^*, uv or u \cdot v is concatenation. Power: for n \in \mathbb{N}, word u, u^n is u concatenating with itself n times. Substring: u is substring of w if w = xuy for some x, y.

// TODO: do exercise proof

Operations on Language L

Size: |L| either finite or countably infinite Reversal: L^R = \{w^R \in \Sigma^* | w \in L\} (reverse every possible word) Concatenation: L_1L_2 = L_1 \cdot L_2 = \{uv \in \Sigma^* | u \in L_1 \land v \in L_2\} (note L_1L_2 \neq L_2L_1) Power: L^n is concatenating L with itself n times. (L^n = \{u_1u_2\cdot \cdot \cdot u_n \in \Sigma^* | \forall i \in \{1, 2, ..., n\} u_i \in L \}) Star: for finite language L, L^* = \bigcup_{n \in \mathbb{N}} L^n = \{u_1u_2 \cdot \cdot \cdot u_n \in \Sigma^* | \forall i \in \{1, 2, ..., n\} n \in \mathbb{N}, u_i \in L\} = L \cup LL \cup LLL \cup ... (Concatenation to itself countable infinite many times)

// TODO: do multiple exercise


Encoding: encoding mapping from a finite or countably infinite set A to \Sigma is \text{Enc} : A \rightarrow \Sigma^*. \langle{a}\rangle is the encoded a \in A.

Valid Encoding: If w \in \Sigma^* \land w = \langle{a}\rangle, then w is a valid encoding of A Encodable: a set that can be encoded (all finite sets are encodable, some infinite sets are encodable by turing machine, undecidable sets are not encodable)

// TODO: exercise

Computational Problems and Decision Problems

Basically, if we can encode something abstract into a common language (coliqual sense, or basis in linear algebra) written in alphabet \Sigma, we can do all the work in that language and translate back.

Computational problem as mapping instances to solutions

Computational problem as mapping instances to solutions

Error String

Error String
Computational Problem: f : \Sigma^* \rightarrow \Sigma^* is a computational problem over \Sigma

Correspondence between decision problems and languages

Correspondence between decision problems and languages
Decision Problem: a function f : \Sigma^* \rightarrow \{0, 1\}

Entscheidungs Problem: if there a finitary procedure to determine the validity of a given mathematical statement?

// TODO: check your understanding

Study Tips

intuition: understand concept formal: formal definition examples: ... counter examples: ... results: theorems associated with concepts


Add is P.R.

\text{𝖺𝖽𝖽}_2=\text{𝖯𝗋𝖾𝖼}[h, g] where \begin{cases}g = P_1^1\\h = S \circ P_2^3\\\end{cases}

Multiply is P.R.

\begin{cases} \text{mult}_2(0, y) = 0\\ \text{mult}_2(x + 1, y) = y + \text{mult}(x, y)\\ \end{cases}

Factorial is P.R.

\begin{cases} \text{fact}(0) = 1\\ \text{fact}(x+1) = (x+1) \cdot \text{fact}(x)\\ \end{cases}

Table of Content