Lecture 004

Ada's Lecture

Computation Intuition: finite answer to infinite number of question.

• an algorithm should be a finite object

• memory should be unbounded (to allow unbounded input)

$\text{Regular Languages} \in \text{Computable}$

Turing Machine

Totally Minimal (TM): programming Language

• can simulate bytecode

• simple to reason about

Turing Machine:

• idea: DFA + infinite tape

  • DFA: accept instructions
    • move head left (transition property)
    • move head right (transition property)
    • write a symbol from alphabet (transition property)
    • if head is reading, GOTO (step / line) (transition property)
    • halt and accept (state property: $q_{rcc}$)
    • halt and reject (state property: $q_{rej}$)
  • infinite tape: $\gamma \in \mathbb{Z}$ (or $\gamma \in \mathbb{N}$ is fine)
    • with input at $w_1...w_n \in \Sigma^*$ where $w_i$ placed on $i-1$
    • two-direction movable pointer (head) at $0$
    • pointer can for read and write
    • other cell blank

• notation: $a \mapsto \sqcup, R$ (change $a$ to $\sqcup$ and go to state $R$)

  • You can leave out notation when going to accept or reject state (because writing and moving pointer is not relevent)
  • You can do $\mapsto L$ is you just want to write the same input (and you promise not to read it again)
  • Omitted information defined arbitrarily

Example of a Turing Machine in Pseudocode

switch (LETTER)
  case 'a': write 'b'; move LEFT;  GOTO STATE 2;
  case 'b': write ' '; move RIGHT; GOTO STATE 0;
  case ' ': write 'b'; move LEFT;  GOTO STATE 1;

Formal Definition of Turing Machine: $M = (Q, \Sigma, \Gamma, \delta, q_0, q_{acc}, q_{rej})$

• $Q$ (set of states): finite, non-empty

• $\Sigma$ (input alphabet): finite, non-empty set with $\sqcup \notin \Sigma$

• $\Gamma$ (tape alphabet): finite set with $\sqcup \in \Gamma \land \Sigma \subset \Gamma$ (it adds no textbook)

• $\delta : Q \times \Gamma \rightarrow Q \times \Gamma \times \{L, R\}$ (transition function): function takes in states and tape alphabet, write tape alphabet, goto state, move pointer.

• $q_0$ (start state): an element of Q

• $q_{acc}$ (accepting state): an element of Q

• $q_{rej}$ (rejecting state): an element of Q $q_{rej} \neq q_{acc}$

Definitions:

• Equivalence: same machine up to renaming the elements of the sets $Q$, $\Sigma$ and $\Gamma$.

• Configuration ($uqv \in (\Gamma \cup Q)^*$): current running $M$ contains current tape ($uv$), current head (at $v_0$), and current state ($q$)

  • where $u, v \in \Gamma \land q \in Q$ ($u$ cannot start with $\sqcup$ and $v$ cannot end with $\sqcup$)
  • tape is $...\sqcup \sqcup uv \sqcup \sqcup ...$
  • where head points to left most symbol of $v$. when $v = =\epsilon$, head is at $\sqcup$ at right of $u$
  • $\alpha \vdash_M \beta$ ($\alpha$ yields $\beta$ (in $M$)): Knowing current configuration $\alpha$ and transition function $\delta$, one can determine the next configuration $\beta$ after a step

• $M(x)$ that halts ($\downarrow$, converges): if there exists a computational trace (sequence of configurations) $\alpha_1\alpha_2...\alpha_T$ such that

  • start from initial state: $\alpha_0 = q_0x$
  • all configuration has a next configuration $\alpha \vdash_M a_{t+1}$ for all $t = 0, 1, 2... T-1$
  • $\alpha_T$ is either accepting configuration or rejecting configuration

• $M(x)$ that diverges ($\uparrow$): loop

• accepting configuration: if $q$ is accept state

• rejecting configuration: if $q$ is rejecting state

• Equivalence of computations: let $x_1x_2 \in \Sigma^*$ be inputs to $((M_1(x_1)\uparrow \land M_2(x_2) \uparrow) \lor (M_1(x_1) \downarrow \land M_2(x_2) \downarrow \land (\text{ two computations return the same result })) \implies M_1(x_1) \simeq M_2(x_2)$

• Turing machines can loop forever: accept, reject, loops forever

DFA limitation:

• no unbounded memory, no access non-input cell (memory does not scale with input length defines regular language, if allow more than log-log memory that scale with input, not DFA anymore)

• only move right (not a problem)

• can't write (inability to write defines DFA)

• DFA halts when all input read!

TM limitation:

• can't have more than one accepting state

• TM might loop forever!

// TODO: Exercise (Practice with configurations) // WARNING: you must do this

Language Accepted: $\mathbb{L}(M)$ is the set of all strings that TM $M$ accepts (not necessarily a decider).

Language Decides: TM (must be a decider) $M$ decides $L \subseteq \Sigma^*$ if $(\forall w \in L \land M \text{ accepts } w) \lor (\forall w \notin L \land M \text{ rejects } w)$

Decider: a TM that halts on all inputs

Decidable Language: if a $L$ is decidable, then $\exists M$ such that $L = \mathbb{L}(M)$ where $M$ is a decider TM

• Decidability is closed under: given $L_1, L_2$ decidable, the following are decidable
  • $L_1L_2$
  • $L_1^*$
  • $L_1 \cup L_2$
  • $L_1 \cap L_2$
  • $\bar{L_1}$

// TODO: Exercise (A simple decidable language) // TODO: Exercise (Drawing TM state diagrams)

Encoding:

• $\hat{M, x}$: encode tuple $(M, x)$ into natural number

• $\langle{M, x}\rangle$: encode tuple $(M, x)$ into string

• $\{\langle{M}\rangle\}$: decode turing machine (or alternative $\varphi_{\langle{M}\rangle}$)

• Index of a machine: The index of a Turing machine $M$ is $e = \hat{M} \in \mathbb{N}$. The decoding is $M = \{e\}$

Property of Turing Machine:

• can shift entire input string

• can convert input from $x_1x_2x_3x_4$ to $x_1\sqcup x_2\sqcup x_3\sqcup x_4$

• can simulate big $\Gamma$ with $\{0, 1, \sqcup\}$

• can mark cells by doing $\Gamma = \{0, 1, 0^*, 1^*, \sqcup\}$

• can copy one region of tape

• can simulate (TM with 2~k tapes and heads) using 1 tape

• can implement data structure, simulate random access memory, simulate assembly language

Theorem: any language can be computed in "Python, Java, C, etc..." can be decided by a TM.

Describe TM:

• Low-level: state diagram

• Medium-level: describe movement of tape head

• High-level: pseudocode, algorithm

Church-Turing Thesis: TM is equivalent of a person doing calculation

Universal Machine: can simulate all other machines (a human is a universal machine) by feeding in both input and instructions since all machine can be encoded

• a turing machine: $U$ takes $\langle{M, x \in \Sigma^*}\rangle$ as input where $M$ is TM

• a interpreter

def U(⟨TM M,string x⟩):

1. Simulate M on input x (𝚒.𝚎. run M(x))
2. If it accpets, accept.
3. If i𝚝 rejects, reject.

Note on actual TM:

• To make sure M always halts, we can add a third input, an integer k, and have the universal machine simulate the input TM for at most k steps.

• We assume TM checks type mismatch before running. Reject if does not match.

Self-referencing: feed machine with its own code can create problem by code is data, leads to undecidability.

(Physical) Church-Turining Thesis: any computational problem that can be solved by any physical process, can be solved by a (probabilistic, or quantum) TM. (Not everybody agrees with this arguments)

• Arguments:
  • Bekenstein bound (space is discrete) gives finite number of states
  • Physics are computable (except maybe randomness?)

Strong Physical Church-Turing Thesis: What can be computed efficiently in this universe by any physical process or device, can be computed efficiently by a TM (False)

Extended Physical Church-Turing Thesis: What can be computed efficiently in this universe by any physical process or device, can be computed efficiently by a Q(Quantum)M (True)

1. Study TMs == Study computational limit of universe
2. computational limit of universe is everywhere
3. You can simulate universe in universe. The universe may be simulation.

Good Paper:

• Black Holes at Exp-Time (Susskind)

• Complexity and Momentum (Susskind, Zhao)

• The Complexity Geometry of a Single Qubit (Brown, Susskind)

• Black Holes and Complexity Classes (Susskind)

Conway's Game of Life: simplest Universe that caputures full computability

• Simulate Turing Machine: https://www.youtube.com/watch?v=My8AsV7bA94

• Simulate Game of Life: https://www.youtube.com/watch?v=xP5-iIeKXE8 (And those cells have no idea that they are in the simulation)

// QUESTION: if a machine has n state, how much bigger can another machine simulate this machine? (This is an insteresting question because we can determinate the maximum granularity of our universe if it is a simulation)

// QUESTION: is there physical bound to simulation so that we can detect if we are in simulation?

Languages related to encodings of DFAs: (all decidable)

• $ACCEPTS_{DFA} = \{\langle{D, x}\rangle: D \text{ is a DFA that accepts string x}\}$

• $SELF-ACCEPTS_{DFA} = \{\langle{D}\rangle: D \text{ is a DFA that accepts string } \langle{D}\rangle\}$

• $EMPTY_{DFA} = \{\langle{D}\rangle: D \text{ is a DFA with } L(D) = \emptyset\}$ (construct DFA's graph and do a graph search from q_0 to F)

• $EQ_{DFA} = \{\langle{D_1, D_2}\rangle: D_1, D_2 \text{ are DFAs with } L(D_1) = L(D_2)\}$ ($EQ_{DFA} \leq EMPTY_{DFA}$, $EQ_{DFA}$ reduce to $EMPTY_{DFA}$ by constructing DFA for $L(D_1) - L(D_2)$ and check whether it is $EMPTY_{DFA}$)

// TODO: Exercise (Practice with decidability through reductions)

// TODO: Check Your Understanding

Sutner's Lecture

Turing Machine

Flaw in Primitive Recursive: no interpreter for primitive recursive functions can be primitive recursive

• AFSOC $f(x) = \text{eval}(x, x) + 1$ is P.R. (where $x$ is a P.R. function, $\text{eval}$ is P.R. interpreter)

• $f(e) = \text{eval}(e, e) + 1 = f(e) + 1$ is not P.R., contradiction

• (for TM, both $f(e)$ and $f(e)+1$ is undefined, so $UND = UND$ is not a problem, by Undefined, Sutner means "does not halt" and "not valuable")

Turing Machine: good model close to register machines or random access machines

• tape: a bi-infinite stripe of paper

• cells: cells on tape

• tape inscription: represent finite word on tape alphabet

• finite state control: head

Busy Beaver Problems: single-digit number of states TM are very hard to analyze (https://mathworld.wolfram.com/BusyBeaver.html)

Turing Computability

Configuration (Instantaneous description, ID): a word $ypx$ where $x, y \in \Sigma^*$ and $p \in Q$

• one step: $ypx \xrightarrow[M]{1} y'qx'$ relation between two configurations

• 

Turing Machine and P.R.

• $(\forall t\in\mathbb{N}, C, C')(R(t, C, C') \iff C \xrightarrow[M]{t} C' \text{ is P.R.})$

• however, if $t = \infty$, then it is no long P.R., it is TM.

• the difference between P.R. and TM computable is one unbounded search, one existential qualifier

Convention: placing init and halt at left of all input/output:

• $C^{\text{init}}_x = q_{init} \sqcup x_1x_2...x_n$

• $C^{\text{halt}}_y = q_{halt} \sqcup y_1y_2...y_m$

• output: $y$ is the output if $C^{\text{init}}_x \xrightarrow[M]{} C^{\text{halt}}_y$

• $M$ computes the partial (not total because potential loop) function $f : \Sigma^* \nrightarrow \Sigma^*$

  • if $f$ defined on $x$ then $C^{\text{init}}_x \xrightarrow[M]{} C^{\text{halt}}_y$ and $f(x) = y$
  • if undefined on $x$, then $M$ does not halt

Computable: partial function $f : \Sigma^* \nrightarrow \Sigma^*$ is (Turing) computable if there is TM $M$ computes $f$

(transducer) TM: partial recursive (computable) function decider (acceptors) TM: total recursive functions

Different Kinds of TM:

• $n$ tape: separate $n$ many read/write head

• $n$ track: read and write tuples of $n$ elements

• one-way infinite: tape space $\mathbb{N}$

• two-way infinite: tape space $\mathbb{Z}$

• read/write only input tape: always one-way infinite

Semidecidable Set: $A \subseteq \mathbb{N}^k$ is simidecidable if there is a TM $M$ that halt precisely on all $x \in A$, and halt on all $x \notin A$.

• Semidecidable Set closed under intersection: run machines sequentially

• Semidecidable Set closed under union: run parallel

• Semidecidable Set NOT CLOSED UNDER COMPLEMENT.

Decidable Set: $A \subseteq \mathbb{N}$ is decidable iff $A$ and $\mathbb{N} - A$ are both semidecidable. (parallel running two machines that are complement of each other)

// QUESTION: understand R.E. Recursive Enumerable (R.E.): if there is a partial TM computable function $f: \mathbb{N} \nrightarrow \mathbb{N}$ such that $A$ is the range of $f$ (intuitively: output that can be generated by a TM, where TM continuously spit out elements $A$. formally: apply all possible $n \in \mathbb{N}$ and the output will be $A$)

• Recognition: define computable in terms of if a computation can recognize whether an element is in the set

• Generation: connect to context-free grammar, define computation as if a computation can generate the set itself

So semidecidable iff recursive enumerable. (prove using dovetailing: run infinitely many computations in parallel using stages)

Comparing partial function: function $\alpha = \beta$ is both functions are defined on input $x$ and $\alpha(x) = \beta(x)$ or none of them are defined on $x$.

Define value of looping function: $f(x) = \lim_{t \rightarrow \infty} f_t(x)$ where $f_{t}(x) = \begin{cases} y \text{ if halt and output y}\\ \uparrow \text{ otherwise}\\ \end{cases}$

• Run all input with finite many step on "Recognition"

• Put the halting result generated to "Generation"

• // QUESTION: understand

Hilbert's Dream: a program spit out all math theorem:

1. enumerate all possible formula (no problem)
2. filter out actual proofs (need a decider for checking formula, but we don't have, since may loop)
3. output filtered result