Computation Intuition: finite answer to infinite number of question.
an algorithm should be a finite object
memory should be unbounded (to allow unbounded input)
\text{Regular Languages} \in \text{Computable}
Totally Minimal (TM): programming Language
can simulate bytecode
simple to reason about
Turing Machine:
idea: DFA + infinite tape
notation: a \mapsto \sqcup, R (change a to \sqcup and go to state R)
Example of a Turing Machine in Pseudocode
switch (LETTER)
case 'a': write 'b'; move LEFT; GOTO STATE 2;
case 'b': write ' '; move RIGHT; GOTO STATE 0;
case ' ': write 'b'; move LEFT; GOTO STATE 1;
Formal Definition of Turing Machine: M = (Q, \Sigma, \Gamma, \delta, q_0, q_{acc}, q_{rej})
Q (set of states): finite, non-empty
\Sigma (input alphabet): finite, non-empty set with \sqcup \notin \Sigma
\Gamma (tape alphabet): finite set with \sqcup \in \Gamma \land \Sigma \subset \Gamma (it adds no textbook)
\delta : Q \times \Gamma \rightarrow Q \times \Gamma \times \{L, R\} (transition function): function takes in states and tape alphabet, write tape alphabet, goto state, move pointer.
q_0 (start state): an element of Q
q_{acc} (accepting state): an element of Q
q_{rej} (rejecting state): an element of Q q_{rej} \neq q_{acc}
Definitions:
Equivalence: same machine up to renaming the elements of the sets Q, \Sigma and \Gamma.
Configuration (uqv \in (\Gamma \cup Q)^*): current running M contains current tape (uv), current head (at v_0), and current state (q)
M(x) that halts (\downarrow, converges): if there exists a computational trace (sequence of configurations) \alpha_1\alpha_2...\alpha_T such that
M(x) that diverges (\uparrow): loop
accepting configuration: if q is accept state
rejecting configuration: if q is rejecting state
Equivalence of computations: let x_1x_2 \in \Sigma^* be inputs to ((M_1(x_1)\uparrow \land M_2(x_2) \uparrow) \lor (M_1(x_1) \downarrow \land M_2(x_2) \downarrow \land (\text{ two computations return the same result })) \implies M_1(x_1) \simeq M_2(x_2)
Turing machines can loop forever: accept, reject, loops forever
DFA limitation:
no unbounded memory, no access non-input cell (memory does not scale with input length defines regular language, if allow more than log-log memory that scale with input, not DFA anymore)
only move right (not a problem)
can't write (inability to write defines DFA)
DFA halts when all input read!
TM limitation:
can't have more than one accepting state
TM might loop forever!
// TODO: Exercise (Practice with configurations) // WARNING: you must do this
Language Accepted: \mathbb{L}(M) is the set of all strings that TM M accepts (not necessarily a decider).
Language Decides: TM (must be a decider) M decides L \subseteq \Sigma^* if (\forall w \in L \land M \text{ accepts } w) \lor (\forall w \notin L \land M \text{ rejects } w)
Decider: a TM that halts on all inputs
Decidable Language: if a L is decidable, then \exists M such that L = \mathbb{L}(M) where M is a decider TM
// TODO: Exercise (A simple decidable language) // TODO: Exercise (Drawing TM state diagrams)
Encoding:
\hat{M, x}: encode tuple (M, x) into natural number
\langle{M, x}\rangle: encode tuple (M, x) into string
\{\langle{M}\rangle\}: decode turing machine (or alternative \varphi_{\langle{M}\rangle})
Index of a machine: The index of a Turing machine M is e = \hat{M} \in \mathbb{N}. The decoding is M = \{e\}
Property of Turing Machine:
can shift entire input string
can convert input from x_1x_2x_3x_4 to x_1\sqcup x_2\sqcup x_3\sqcup x_4
can simulate big \Gamma with \{0, 1, \sqcup\}
can mark cells by doing \Gamma = \{0, 1, 0^*, 1^*, \sqcup\}
can copy one region of tape
can simulate (TM with 2~k tapes and heads) using 1 tape
can implement data structure, simulate random access memory, simulate assembly language
Theorem: any language can be computed in "Python, Java, C, etc..." can be decided by a TM.
Describe TM:
Low-level: state diagram
Medium-level: describe movement of tape head
High-level: pseudocode, algorithm
Church-Turing Thesis: TM is equivalent of a person doing calculation
Universal Machine: can simulate all other machines (a human is a universal machine) by feeding in both input and instructions since all machine can be encoded
a turing machine: U takes \langle{M, x \in \Sigma^*}\rangle as input where M is TM
a interpreter
def U(⟨TM M,string x⟩):
1. Simulate M on input x (𝚒.𝚎. run M(x))
2. If it accpets, accept.
3. If i𝚝 rejects, reject.
Note on actual TM:
To make sure M always halts, we can add a third input, an integer k, and have the universal machine simulate the input TM for at most k steps.
We assume TM checks type mismatch before running. Reject if does not match.
Self-referencing: feed machine with its own code can create problem by code is data, leads to undecidability.
(Physical) Church-Turining Thesis: any computational problem that can be solved by any physical process, can be solved by a (probabilistic, or quantum) TM. (Not everybody agrees with this arguments)
Strong Physical Church-Turing Thesis: What can be computed efficiently in this universe by any physical process or device, can be computed efficiently by a TM (False)
Extended Physical Church-Turing Thesis: What can be computed efficiently in this universe by any physical process or device, can be computed efficiently by a Q(Quantum)M (True)
Good Paper:
Black Holes at Exp-Time (Susskind)
Complexity and Momentum (Susskind, Zhao)
The Complexity Geometry of a Single Qubit (Brown, Susskind)
Black Holes and Complexity Classes (Susskind)
Conway's Game of Life: simplest Universe that caputures full computability
Simulate Turing Machine: https://www.youtube.com/watch?v=My8AsV7bA94
Simulate Game of Life: https://www.youtube.com/watch?v=xP5-iIeKXE8 (And those cells have no idea that they are in the simulation)
// QUESTION: if a machine has n state, how much bigger can another machine simulate this machine? (This is an insteresting question because we can determinate the maximum granularity of our universe if it is a simulation)
// QUESTION: is there physical bound to simulation so that we can detect if we are in simulation?
Languages related to encodings of DFAs: (all decidable)
ACCEPTS_{DFA} = \{\langle{D, x}\rangle: D \text{ is a DFA that accepts string x}\}
SELF-ACCEPTS_{DFA} = \{\langle{D}\rangle: D \text{ is a DFA that accepts string } \langle{D}\rangle\}
EMPTY_{DFA} = \{\langle{D}\rangle: D \text{ is a DFA with } L(D) = \emptyset\} (construct DFA's graph and do a graph search from q_0 to F)
EQ_{DFA} = \{\langle{D_1, D_2}\rangle: D_1, D_2 \text{ are DFAs with } L(D_1) = L(D_2)\} (EQ_{DFA} \leq EMPTY_{DFA}, EQ_{DFA} reduce to EMPTY_{DFA} by constructing DFA for L(D_1) - L(D_2) and check whether it is EMPTY_{DFA})
// TODO: Exercise (Practice with decidability through reductions)
// TODO: Check Your Understanding
Flaw in Primitive Recursive: no interpreter for primitive recursive functions can be primitive recursive
AFSOC f(x) = \text{eval}(x, x) + 1 is P.R. (where x is a P.R. function, \text{eval} is P.R. interpreter)
f(e) = \text{eval}(e, e) + 1 = f(e) + 1 is not P.R., contradiction
(for TM, both f(e) and f(e)+1 is undefined, so UND = UND is not a problem, by Undefined, Sutner means "does not halt" and "not valuable")
Turing Machine: good model close to register machines or random access machines
tape: a bi-infinite stripe of paper
cells: cells on tape
tape inscription: represent finite word on tape alphabet
finite state control: head
Busy Beaver Problems: single-digit number of states TM are very hard to analyze (https://mathworld.wolfram.com/BusyBeaver.html)
Configuration (Instantaneous description, ID): a word ypx where x, y \in \Sigma^* and p \in Q
one step: ypx \xrightarrow[M]{1} y'qx' relation between two configurations
Turing Machine and P.R.
(\forall t\in\mathbb{N}, C, C')(R(t, C, C') \iff C \xrightarrow[M]{t} C' \text{ is P.R.})
however, if t = \infty, then it is no long P.R., it is TM.
the difference between P.R. and TM computable is one unbounded search, one existential qualifier
Convention: placing init and halt at left of all input/output:
C^{\text{init}}_x = q_{init} \sqcup x_1x_2...x_n
C^{\text{halt}}_y = q_{halt} \sqcup y_1y_2...y_m
output: y is the output if C^{\text{init}}_x \xrightarrow[M]{} C^{\text{halt}}_y
M computes the partial (not total because potential loop) function f : \Sigma^* \nrightarrow \Sigma^*
Computable: partial function f : \Sigma^* \nrightarrow \Sigma^* is (Turing) computable if there is TM M computes f
(transducer) TM: partial recursive (computable) function decider (acceptors) TM: total recursive functions
Different Kinds of TM:
n tape: separate n many read/write head
n track: read and write tuples of n elements
one-way infinite: tape space \mathbb{N}
two-way infinite: tape space \mathbb{Z}
read/write only input tape: always one-way infinite
Semidecidable Set: A \subseteq \mathbb{N}^k is simidecidable if there is a TM M that halt precisely on all x \in A, and halt on all x \notin A.
Semidecidable Set closed under intersection: run machines sequentially
Semidecidable Set closed under union: run parallel
Semidecidable Set NOT CLOSED UNDER COMPLEMENT.
Decidable Set: A \subseteq \mathbb{N} is decidable iff A and \mathbb{N} - A are both semidecidable. (parallel running two machines that are complement of each other)
// QUESTION: understand R.E. Recursive Enumerable (R.E.): if there is a partial TM computable function f: \mathbb{N} \nrightarrow \mathbb{N} such that A is the range of f (intuitively: output that can be generated by a TM, where TM continuously spit out elements A. formally: apply all possible n \in \mathbb{N} and the output will be A)
Recognition: define computable in terms of if a computation can recognize whether an element is in the set
Generation: connect to context-free grammar, define computation as if a computation can generate the set itself
So semidecidable iff recursive enumerable. (prove using dovetailing: run infinitely many computations in parallel using stages)
Comparing partial function: function \alpha = \beta is both functions are defined on input x and \alpha(x) = \beta(x) or none of them are defined on x.
Define value of looping function: f(x) = \lim_{t \rightarrow \infty} f_t(x) where f_{t}(x) = \begin{cases} y \text{ if halt and output y}\\ \uparrow \text{ otherwise}\\ \end{cases}
Run all input with finite many step on "Recognition"
Put the halting result generated to "Generation"
Hilbert's Dream: a program spit out all math theorem:
Table of Content