\documentclass[12pt]{article}
\usepackage{amssymb, amsmath}

\textheight 23cm
\textwidth 6.5in
\oddsidemargin -.1in
\evensidemargin -.1in
\topmargin -1.5cm

\renewcommand{\theenumi}{\arabic{enumi}}
\renewcommand{\labelenumi}{\theenumi)}

\newtheorem{exer}{Exercise}

\newcommand{\eqsyn}{=_{syn}}
\newcommand{\ra}{\rightarrow}
\newcommand{\SL}{{\cal L}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\underN}{\underline{\mathbb{N}}}
\newcommand{\stand}{\underline{\mathbb{N}}}
\newcommand{\calM}{{\cal M}}
\newcommand{\calL}{{\cal L}}
\newcommand{\calP}{{\cal P}}
\newcommand{\ld}{{\overline{\:.\,}}}
\newcommand{\dl}{{\frac{\:.\:}{}}}
\newcommand{\emp}{\varnothing}


\def\Konig{K\"{o}nig}
\def\Godel{G\"{o}del}
\def\Equiv{\Longleftrightarrow}

\input{prooftree}

\parskip 2ex
\parindent 0ex

\setcounter{page}{54}

\begin{document}

{\large\bf CS 4995}\hspace{.8in}{\large\bf Notes} (S. Cook and T. Pitassi)
\hfill {\large\bf Fall, 2022}

\bigskip

\begin{center}
{\Large\bf Computability Theory}
\end{center}

This section is partly
inspired by the material in ``A Course in Mathematical Logic''
by Bell and Machover, Chap 6, sections 1-10.\\
Other references: ``Introduction to the theory of computation'' by
Michael Sipser, and ``Computability, Complexity, and
Languages'' by M. Davis and E. Weyuker.

Our first goal is to give a formal definition for what it means
for a function on $\NN$ to be computable by an algorithm.
Historically the first convincing such definition was given by
Alan Turing in 1936, in his paper which introduced what we now
call Turing machines.  Slightly before Turing, Alonzo Church
gave a definition based on his lambda calculus.  About the same
time \Godel, Herbrand, and Kleene developed definitions based
on recursion schemes.  Fortunately all of these definitions are
equivalent, and each of many other definitions proposed later
are also equivalent to Turing's definition.  This has lead to
the general belief that these definitions have got it right, and
this assertion is roughly what we now call ``Church's Thesis''.


A natural definition of computable function $f$ on $\NN$ allows for
the possibility that $f(x)$ may not be defined for all $x\in\NN$,
because algorithms do not always halt.  Thus we will use the
symbol $\infty$ to mean ``undefined''.

{\bf Definition:}
A {\em partial function} is a function
$$f : ({\NN} \cup \{ \infty \})^n \rightarrow {\NN} \cup \{ \infty \}, n\ge 0$$ 
such that
$f(c_1,...,c_n) = \infty$ if some $c_i=\infty$. 

In the context of computability theory, whenever we refer to a
function on $\NN$, we mean a partial function in the above sense.

{\bf Definitions:}
$$ \mbox{Domain$(f) = \{ \vec{x} \in {\NN}^n \mid f(\vec{x}) \neq \infty \}$} $$
where $\vec{x} = (x_1 \cdots x_n)$
We say $f$ is {\em total} iff Domain$(f) = {\NN}^n$ (i.e. if $f$ is always
defined when all its arguments are defined).

\bigskip

{\bf Turing machines}

A Turing Machine is specified by a 7-tuple, $M = \{ Q, \Sigma, \Gamma, \delta, q_1, B, \{q_2\} \},$
where $Q = \{q_1, q_2, \ldots, q_k\}$ is a finite set of states,
$\Sigma$ is a finite input alphabet, including the two elements $0$ and $1$;
$\Gamma$ is a finite tape alphabet, such that $\Sigma \subseteq \Gamma$;
$q_1$ is the designated start state, $q_2$ is the designated halt state,
and $B \in \Gamma$ is a special blank symbol not in $\Sigma$.
Lastly, the transition function, $\delta$, is a function from 
$Q \times \Gamma$ to $Q \times \Gamma \times \{L,R\}$.

Let $\Sigma$ be a finite alphabet.
$\Sigma^*$ denotes the set of all finite length strings over $\Sigma$.
Let $x \in \Sigma^*$.
We visualize a Turing machine $M$ over $\Sigma$ on input $x$ as consisting
of an tape consisting of an infinite sequence of ${\it cells}$, $c_0, c_1, \ldots,$.
There is a tape ${\it head}$ which points to one cell of the tape, and
at every point in time, the Turing machine is in one state $q_i \in Q$.
Each cell of the tape contains an element from $\Gamma$.
$M$ on input $x$ operates as follows.
Initially $M$ is in the start state $q_1$, and the infinite tape
consists of $x$ written on the first $|x|$ consecutive cells, followed by all blank symbols ($B$).
At every time step, $M$ makes one transition, according to $\delta$.
If $M$ is in state $q$ and the tape head is currently reading symbol $s$ at time $t$,
and $\delta(q,s) = (q',s',L/R)$, then at time $t+1$, the new state is $q'$,
the symbol $s$ is replaced by $s'$, and the tape head moves left/right one cell.
(If the tape head is already at the leftmost position, then on a L move,
the head stays in place.)
The computation terminates if the current state of $M$ is
the halt state, $q_2$.
When $M$ halts, define $y$ to be the longest string on the tape such that $y$ contains
no blank ($B$) symbols. Then $M$ on $x$ outputs $y$.


%A Turing Machine accepts a language, $L \subseteq \Sigma^*$ if and only if
%for every $x \in L$, $M$ on $x$ accepts, and for every $x \not \in L$,
%$M$ on $x$ does not accept. 

Turing machines compute $n$-ary partial (or total) functions
from $\NN ^n$ to $\NN$ by encoding the input tuples and outputs as strings
over $\Sigma$. 
First, we will assume that each number in $\NN$ is written in binary.
We will encode an $n$-tuple $a_1, \ldots, a_n$ by a string $x$
where $x$ consists of the concatenation of $a_1,\ldots,a_n$ (written in
binary), with each $a_i$ separated by
the symbol "2". Let $<a_1,\ldots,a_n>$ denote the encoding of
$a_1,\ldots,a_n$. Let $f$ be an $n$-ary total function. 
$M$ computes the function $f$ if for every $n$-tuple
$a_1,\ldots,a_n$, $M$ on input $<a_1,\ldots,a_n>$ outputs $y = f(a_1..a_n)$
(again written in binary).
In such a case, $f$ is called a {\it total} {\it computable} function.

If $f$ is a partial function, 
then $M$ computes $f$ if for all tuples $a_1,\ldots,a_n$ in the
domain of $f$, $M$ on input $<a_1,\ldots,a_n>$ outputs $f(a_1,\ldots,a_n)$.
Note that on inputs not in the domain of $f$, $M$ may not halt.
In such a case, $f$ is called a {\it computable} {\it partial} function.



\label{CT}
Our form of {\bf Church's Thesis:}\\ 
\mbox{}\hspace{.5in} Every algorithmically computable function is
TM-computable.

Here the notion ``algorithmically computable'' is not a precise
mathematical notion, but rather an intuitive notion.
It is understood that the algorithms in question have unlimited
memory.  In the case of register machines, this means that
each register can hold an arbitrarily large natural number.
It is a strong claim about the robustness of our formal notion of computable
function.  In general, if we give an informal algorithm to compute
a function, then we can claim that it is computable, by Church's
Thesis.

Alonzo Church proclaimed this famous ``thesis'' in a footnote to a
paper in 1936.  Actually, he did not talk about TM's, but rather
claimed that every algorithmically computable function is definable
using the $\lambda$-calculus which he had invented.   A little later
Alan Turing published his famous paper defining what are now called
Turing machines, and argued, more convincingly than Church,
that every algorithmically computable
function is computable on a Turing machine.  (Hence ``Church's
Thesis'' is sometimes called the ``Church-Turing thesis''.)  Turing
proved that Church's $\lambda$-definable functions coincide with the
Turing computable functions.  
In fact, many other formalisms for defining algorithmically
computable functions have been given, and all of them turn out to
be equivalent.  This robustness is a powerful argument in favour
of Church's thesis.



\begin{exer}
Write Turing machine programs to compute each of the following
functions:

$f_1(x) =  x + 1$\\
$f_2(x,y)=x\cdot y$

Be sure to respect our input/output conventions for TM's.
\end{exer}
\bigskip

Let $R \subseteq {\Bbb N}^n$.  Thus $R$ is an $n$-ary relation (predicate).
We will think of $R$ as a total 0-1 valued function as follows:
$R(\vec{x})=0$ if and only if $\vec{x} \in R$, and $R(\vec{x})=1$ otherwise.











{\bf Configurations }

We can describe the computation of $M$ on $x$ at time $t$ by a 
{\it configuration}. 
Let $m$ be the leftmost tape cell such that all cells to the right of $q$
contain the blank symbol, $B$.
Then the configuration consists of the contents of the tape up to
cell $m$, plus the current state, $q$, plus the location, $i$, of the tape head.
We will represent this information
by the string $s_1, s_2, ...,(q,s_i),s_{i+1}, \ldots, s_m$,
where $s_1,\ldots,s_q$ are the contents of the tape cells up to cell $m$,
and where $q$ is the current state. (The location of $q$ within the
string tells us the location of the head.)
The initial configuation of $M$ on $x$ is therefore the string
$(q_1, x_1), x_2, \ldots, x_{n-1}, x_n$.


The computation of $M$ on $x$ is described by a sequence
of configurations, $c_1, c_2, \ldots $,
where $c_1$ is the initial configuration of $M$ on $x$,
and such that each $c_i$ follows from the previous configuration
$c_{i-1}$ by applying the transition function $\delta$.
If $M$ halts on $x$, then this sequence of configurations is
finite.
If $M$ halts on $x$ in $m$ steps, we
will visualize the corresponding sequence of configurations
as a {\it tableaux}, or $m-by-m$ matrix, where the first
row of the matrix is the start configuration, and row $i$ is
the configuration at time $t$.


{\bf Encoding Turing Machines }

We want to associate a number with each Turing machine over the
input alphabet $\Sigma = \{0,1,2\}$.
Here is one way to do this.

Our convention is that the states of a TM are always
called $Q=\{q_1, q_2, \ldots, q_n\}$, where $q_1$ is
always the start state, and $q_2$ is always the halt state.
Similarly, assume that the tape symbols are $\Gamma = \{x_1,x_2,\ldots,x_k\}$
where $x_1=0$, $x_2=1$, $x_3 =2$ and $x_4 = B$.
Let "left" be denoted by $D_1$ and let "right" be denoted by $D_2$.
Then we represent the transition $\delta(q_i,x_j) = (q_k,x_l,D_m)$ by
$0^i 1 0^j 1 0^k 1 0^l 1 0^m$.
The code for $M$ is:
$111 code_1 11 code_2 11 \ldots 11 code_r 111,$
where $code_i$ is the code for one of the possible transitions.

{\bf Example}
Let $M = (Q = \{q_1,q_2,q_3\},\Sigma=\{0,1\}, \Gamma = \{0,1,B\}, \delta, q_1, B, \{q_2\} ).$
Below we describe the transitions and their corresponding codes:
\begin{itemize}
\item $\delta(q_1,1) = (q_3,0,R)$, $c_1 = 0^1 1 0^2 1 0^3 1 0 1 0^2$
\item $\delta(q_3,0) = (q_1,1,R)$, $c_2 = 0^3 1 0 1 0 1 0^2 1 0^2 $
\item $\delta(q_3,1) = (q_2,0,R)$, $c_3 = 0^3 1 0^2 1 0^2 1 0 1 0^2$
\item $\delta(q_3,B) = (q_3,1,L)$, $c_4 = 0^3 1 0^4 1 0^3 1 0^2 1 0$
\end{itemize}
Thus $M$ is encoded by the string $111c_1 11 c_2 11 c_3 11 c_4 111$,
and the pair $(M,1011)$ is encoded by the string
$111 c_1 11 c_2 11 c_3 11 c_4 111 1011$.
Note that given an encoding $\#(M,w)$, we can recover the associated
Turing machine $M$ and string $w$.

{\bf A Universal Turing Machine}

A Universal Turing function $U$ takes as input a number 
$\#(M,x)$ where $M$
is a Turing machine. $U$ on input $\#(M,x)$ outputs $y$
if $M$ on input $x$ outputs $y$.
Note that if $M$ does not halt on $x$, then $U$ is
undefined on $x$.


\begin{theorem}
$U$ is a computable function. 
\end{theorem}

\begin{proof}

We will describe a 3-tape TM, $M_U$.
It is a well-known result that a k-tape Turing machine can be simulated
by a 1-tape Turing machine, and thus $U$ is a computable relation.
First $M_U$ checks to see if $\#(M,x)$ is a valid encoding of a Turing machine, $M$,
and if $x$ is of the proper format.
If not, then $M_U$ does not halt.
Otherwise, on input $\#(M,{x})$, $M_U$ will simulate the computation of $M$ on ${x}$.
The first tape of $M_U$ contains the code of $M$.
The second tape of $M_U$ will contain the contents of the tape of TM $M$ as it is
being simulated on input ${x}$.
The third tape of $M_U$ will contain state information.

\begin{itemize}
\item Initially, $\#(M,x)$ is on tape 1.
\item Check tape 1 to make sure it is a valid input. That is, valid codes for $M$
begin and end with "111" and each code $code_i$ is separated by $11$, and
the transitions are of the form $0^i 1 0^j 1 0^k 1 0^l 1 0^m$, where $m = 1$ or $2$.
\item Initialize tape 2 to contain $\$x$.
\item Initialize tape 3 to contain $\$0$ (this is the start state, $q_1$, in unary).
\item Initialize tape 1 to hold $11\$ code_1 11 code_2 11 \ldots 11 code_r 111$.
(a) If tape 3 holds $\$000$ (halt state $q_2$), halt and output $y$,
where $y$ is the output of $M$ when it halts;
(b) otherwise simulate the next step as follows.
Let $x_j$ be the symbol scanned by head 2 and let $0^i$ be
the contents of tape 3. Scan tape 1 from $\$ $ to $111$, looking for a string
beginning with $110^i 1 0^j 1$. If no such string is found, halt and output 0 (reject).
If such a string is found, say it is $110^i 1 0^j 1 0^k 1 0^l 1 0^m$.
Put $0^k$ on tape 3, and write $x_l$ on the tape cell scanned by head 2, and
then move that head in direction $D_m$ one cell.
\end{itemize}




\end{proof}





{\bf Notation:} $\{z\}$ = the program $\calP$ s.t. $\#(\calP) = \hat{z}$

Thus $\{z\} = \left\{ \begin{array}{ll}
 \mbox{the program $P$ such that $\#(P) =z$ if $P$ exists}\\
 \mbox{$\Lambda$ (empty program) otherwise}
\end{array} \right.$

$\{z\}_n$ is the $n$-ary function computed by the program $\{z\}$.

\bigskip



\begin{center}
{\Large\bf Recursive and Recursively Enumerable Sets}
\end{center}

{\large\bf Recursive Sets}

\medskip

For this section, a {\em set} means a subset of ${\mathbb N}^n$, where
usually $n=1$.   Thus formally a set is the same thing as a relation,
which is the same as a total 0-1 valued function.  Thus if
$A \subseteq {\Bbb N}^n$, then we write
$$ A(\vec{x}) = \left\{ \begin{array}{ll}
 0 & \mbox{if $\vec{x} \in A$} \\
 1 & \mbox{otherwise}
\end{array} \right. $$

{\bf Definition:} A set (or relation) is {\em recursive}
(or {\em computable} or {\em decidable})
if it is computable as a total 0-1 valued function.

%NOTE:  The terms ``recursive'' and ``computable'' are interchangeable
%in this section, whether they are applied to functions or sets.
%This makes sense, since we proved that the
%computable functions are the same as the recursive functions
%(see the theorem on page 61 and the corollary to the Kleene Normal
%Form Theorem on page 69).

By Church's thesis, a set $A$ is recursive iff there is an algorithm which,
given $\vec{x}$, determines whether $\vec{x}\in A$. 
(The algorithm must halt on all inputs.)

{\bf Proposition:}  The class of recursive subsets of ${\Bbb N}^n$
is closed under the operations $\cup,\cap,$ complement.

{\bf Proof:}  This is the same as saying that the class of recursive
$n$-ary relations is closed under the Boolean operations $\wedge,\vee,\neg$
(see Lemma, page 62).   $\Box$

\label{second_prop}
{\bf Proposition:} If $R(\vec{x},y)$ is a recursive relation, and
$f(\vec{x})$ is a total computable function, then the relation
$S(\vec{x})=R(\vec{x},f(\vec{x}))$ is a recursive relation.

{\bf Proof:}.  The class of
total computable functions is closed under composition.
$\Box$ 

Note that the assumption that $f$ is total is necessary in the above
proposition, since by definition a recursive relation must be a {\bf total}
0-1 valued function.

We are interested in proving that certain sets are {\em not} recursive.
The standard example is the diagonal halting set $K$.

{\bf Notation}: $K = \{ x \mid \{x\}_1 (x) \neq \infty \}$

Recall that $ \{x\}_1$ is the unary function computed by the program
(coded by) $x$ .  Thus

$K(x) = \left\{ \begin{array}{l}
 \mbox{1 (true) if program $x$ halts on input $x$} \\
 \mbox{0 (false) otherwise}
\end{array} \right. $

Note that $K$ is a version of the famous ``halting problem'',
originally formulated by Alan Turing in the context of
Turing machines.

{\bf Theorem}: $K$ is not recursive.

{\bf Proof:} The proof is a combination of a ``diagonal argument'' and
a reduction.  First the diagonal argument.

Let $\phi_n(x) = \{n\}_1(x)$
for $n=0,1,2,..$.  That is, $\phi_n$ is
the (partial) function of one variable computed by program $\{n\}$.
Thus $\phi_0,\phi_1,...$ is an enumeration of all computable functions
of one variable.  We can list all values of all these functions in
an infinite table, whose $n$-th row is a list of the successive values
$\phi_n(0),\phi_n(1),...$ of the function $\phi_n$.  We now define
a ``diagonal function'' $D(x)$ by making $D(n)$ defined iff $\phi_n(n)$
is undefined.  That is,

$D(x) = \left\{ \begin{array}{l}
 \mbox{0 if $x \not\in K$} \\
 \infty \mbox{ if $x \in K$}
\end{array} \right. $

The list of values of $D(0),D(1),...$ can be obtained by going down the main
diagonal of the above table and changing each $\infty$ to 0 and changing
each defined value to $\infty$.  Thus it is clear that this list of
values cannot coincide completely with any row in the table, because
the $n$-th value in the list disagrees with the $n$-th row at position
$n$.  It follows that $D$ is not a computable function.

More formally, we can prove that $D$ is not computable by contradiction.
If $D$ is computable, then $D=\{e\}_1$ for some $e\in\NN$.  But then

$\{e\}_1 (e) = D(e) = \left\{ \begin{array}{ll}
 \mbox{0 if $e \not\in K$} & \mbox{i.e. $\{e\}_1 (e) = \infty$} \\
 \infty \mbox{ otherwise} & \mbox{i.e. $\{e\}_1 (e) \neq \infty$}
\end{array} \right. $

i.e.\ $\{e\}_1(e)$ is defined iff $\{e\}_1(e)$ is not defined,
a contradiction.   Hence $D$ is not computable.

Now comes the reduction:  We can reduce the computation of $D$ to the
computation of $K$, so that if $K$ is computable then $D$ is computable.
But we just showed that $D$ is not computable, so $K$ is not computable.


{\large\bf Reducibility}

{\bf Definition:} Suppose $A,B \subseteq {\Bbb N}$. Then $A \leq_m B$ ($A$ is 
many-one reducible to $B$) iff there is a total recursive function
$f : {\Bbb N} \rightarrow {\Bbb N}$, such that 
$x \in A \Leftrightarrow f(x) \in B$, for all $x \in {\Bbb N}$.

Note that $\leq_m$ is similar to the notion of $\leq_p$ of
polynomial time reducibility.  The difference is that for the
latter we require that the function $f$ be polynomial time computable.

{\bf Proposition:}  The relation $\leq_m$ is transitive.  That is,
if $A\leq_m B$ and $B\leq_m C$ then $A\leq_m C$.

\begin{exer}
Prove the above proposition.
\end{exer}

{\bf Proposition}: If $A \leq_m B$ and $B$ is recursive then $A$ is recursive

{\bf Proof:} $A(x) = B(f(x))$. $\Box$

Application: To show that $B$ is not recursive, it suffices to show that
$K \leq_m B$.

Example: Let $H = \{ x \mid \{x\}_1 (0) \not= \infty \}$  Thus $x\in H$
iff program $\{x\}$ halts on input 0.

Claim: $H$ is not recursive

Proof: It suffices to show $K \leq_m H$.
Thus we want a total computable $f$ so $x \in K$ iff $f(x) \in H$.
That is, $\{x\}_1 (x) \neq \infty$ iff $\{f(x)\}_1 (0) \neq \infty$

What is the program $\{f(x)\}$?
Program $\{f(x)\}$ on any input $y$ simulates program $\{x\}$ on input $x$.

From the point of view of the program $\{f(x)\}$, $x$ is a constant;
say $x=x_0$.   
Since there is an easy algorithm that transforms $x_0$ to the program
$\{f(x_0)\}$, it follows from Church's thesis that
the function $f$ is computable (i.e. recursive). 

%\begin{exer}
%Show that $f$ above is in fact primitive recursive.  Do this by
%giving an explicit definition of $f(x)$, and use the techniques on
%pages 61-66.
%\end{exer} 

%It turns out that reductions such as the one above can be made easier
%by using a special case of the so-called S-m-n theorem.
%
%{\bf Theorem:} (Special case of the S-m-n theorem)\\
%Let $g(x,y)$ be computable. Then there exists a total
%computable function $f : {\Bbb N} \rightarrow {\Bbb N}$ such
%that $\{ f(x) \}_1 (y) = g(x,y)$, for all $x,y$.
%
%Intuitively this theorem tells us that if $g(x,y)$ is computable, then
%for each fixed value $x=x_0$ of the first argument, $g(x_0,y)$ becomes
%a computable function $g_{x_0}(y)$ of $y$ alone.
%The function $f(x_0)$ tells us the code for the program which computes
%this function $g_{x_0}$.


\begin{exer}
Show that the following sets are not
computable.
Note that it suffices to show 
that the complementary set is not computable, since a set $A$ is
computable iff $A^c$ is computable.
\end{exer}

$A_1=\{x \mid \{x\}_1(5) = \infty \}$\\
$A_2=\{x \mid \mbox{ran}(\{x\}_1) = {\Bbb N} \}$, where
$\mbox{ran}(f) = f({\Bbb N})$ = range of $f$\\
$A_3=\{x \mid \mbox{dom}(\{x\}_1)\ \mbox{is\ finite} \}$, where
dom$(f)=\{x\mid f(x) \neq \infty\}$ is the domain of $f$.

{\bf Rice's Theorem} 

It turns out that the noncomputability
of all of the above examples, and many more, follow from a single
result, known as Rice's Theorem.  We say that $A\subseteq \mathbb{N}$
is a {\em function index set} if for all $e\in A$, if $\{e\}_1=\{e'\}_1$
then $e'\in A$.  Thus if $A$ contains a code for a program that
computes a unary function $\phi$, then $A$ must contain codes for
all programs that compute $\phi$.  We can think of a function
index set as a set of computable functions rather than a set
of numbers.

Note that each of the three sets $A_1,A_2,A_3$ in the above
exercise is a function index set.

{\bf Theorem:} (Rice)  If $A$ is a function index set and $A\ne\varnothing$
and $A\ne \mathbb{N}$ then $A$ is not computable.

\begin{exer}
Prove Rice's Theorem.  Use the same techniques that you used to
prove $A_1,A_2,A_3$ are not computable. (Hint:  First consider the case
in which no code for the empty function $Empt$ (which has empty domain)
is in $A$.)

\end{exer}

\begin{exer}
You may use Church's Thesis in answering the following questions.
That is, to justify that a particular function is computable
is suffices to give an algorithm for computing it.

Define the relation $R(x,y)$ by the condition $R(x,y)$ holds iff
at some time during the computation of program $\{x\}$
on input $0$, 
the first $|y|$ symbols of the tape
contain $y$.  Prove that $R(x,y)$ is not recursive.

\end{exer}
 

\bigskip
{\large\bf Recursively Enumerable Sets}

\bigskip
{\bf Definition:} If $A \subseteq {\Bbb N}^n$ then $A$ is r.e.
({\em recursively enumerable (r.e.)}, or {\em semidecidable} or 
{\em semidecidable})
if there exists a recursive relation $R \subseteq {\Bbb N}^{n+1}$ such that
$$\vec{x} \in A \Leftrightarrow \exists y R(\vec{x}, y), \quad
       \mbox{for all } \vec{x} \in {\Bbb N}^n$$

{\bf Intuition}: Let $n=1$. $A$ is r.e.\ iff there is an algorithm for
enumerating members of $A$ in some order.  The following Lemma justifies
this intuition.

{\bf Intuition}: Recall that $A$ is recursive (decidable) if there is a TM $M$ that
always halts and such that for all $x \in A$, $M$ halts and outputs 1 (accepts),
and for all $x \not \in A$, $M$ halts and outputs 0 (rejects).
In contrast, $A$ is r.e. if there is a TM $M$ such that for all $x \in M$,
$M$ halts and outputs 1 (accepts), and for all $x \not \in M$, 
$M$ either does not halt, or halts and does not output 1 (halts and
does not accept). Thus any $A$ that is recursive is also r.e., but the
converse does not necessarily hold.

{\bf Lemma:} If $A \subseteq {\Bbb N}$ then $A$ is r.e.\ iff $A = \emp$ or
$A = \mbox{ran}(f)$
for some total computable $f : {\Bbb N} \rightarrow {\Bbb N}$.

If $A = \mbox{ran}(f)$, then $A = \{ f(0), f(1), f(2) \cdots \}$.
Hence there is an algorithm for enumerating $A$, namely
compute $f(0), f(1), f(2) \cdots$.  It is important that $f$ be total
in order for this algorithm to work.   Notice that this does not necessarily 
enumerate $A$ in order, and there may be repetitions.

{\bf Proof of Lemma:} $\Rightarrow$:  Suppose
$x \in A \Leftrightarrow \exists y R(x,y)$.
We want a total computable $f$ so $A = \mbox{ran}(f)$.
Idea: Enumerate all pairs $\langle x,y \rangle$.  We may assume
$A \neq \emp$, so let $a\in A$.  First we define a total computable
function $F(x,y)$ of two variables whose range is $A$:
$$F( x,y ) = \left\{ \begin{array}{ll}
 x & \mbox{if $R(x,y)$ holds} \\
 a & \mbox{otherwise}
\end{array} \right.$$ 
Then $A = \mbox{ran}(F)$.  To convert $F$ to a unary function $f$
with the same range, we fix things so that $f(2^x3^y) = F(x,y)$.
Explicitly, define $f(z)= F((z)_0, (z)_1)$, where $(z)_x$ is the
exponent of prime $p_x$ in the prime decomposition of $z$
Thus $f$ is a total computable unary function
whose range is $A$.

Proof of direction $\Leftarrow$:  Suppose $A=\mbox{ran$(f)$}$, where
$f:\NN \rightarrow \NN$ is a total computable function.  Define the
relation $R(x,y)$ by
$$   R(x,y) = (x = f(y))  $$
Then $R(x,y)$ is recursive.
% because it results from substituting
%a total computable function $f(y)$ for $z$ in the recursive relation $(x=z)$.
%(See the second Proposition on page \pageref{second_prop}.)  
Now it is clear that
$$  x\in A \Leftrightarrow \exists y (x=f(y)) \Leftrightarrow
                \exists y R(x,y)  \qquad \Box $$
%$\Box$

The technique used in the first half of the above proof
of enumerating $A$ by, in effect, enumerating
all pairs $(x,y)$ is called {\em dovetailing}.

{\bf Remark:} Every recursive set is r.e.  Given a recursive set $A$,
simply define the relation
$R$ by $R(x,y) \Leftrightarrow x \in A$.  Then 
$x\in A \Leftrightarrow \exists yR(x,y)$, so $A$ is r.e.

The converse is false, as we shall soon see.

{\bf Analogy:}  P is to NP as the recursive sets are to the r.e. sets.
In fact, one way to define NP is to modify our definition of r.e.
by requiring the relation $R(x,y)$ be polynomial time computable
(instead of just recursive), and by putting a suitable bound
on the quantifier $\exists y R(x,y)$.  Then P is a subset of NP
just as every recursive set is r.e.  However, unlike P vs NP we
can prove that not all r.e. sets are recursive.

{\bf Theorem:}  $K$ is r.e but not recursive.

{\bf Proof:}  Recall that $K=\{x\mid \{x\}_1(x)\neq\infty\}$.  We have
already shown that $K$ is not recursive, so it suffices to show
that $K$ is r.e.   
We will modify our Universal Turing machine, $M_U$, as follows.
On input $x$, we construct $\{x\}$, the program encoded by $x$,
and simulate $\{x\}$ on input $x$.
If the simulation halts and outputs $1$, then we
halt and output 1; if the simulation halts and does not output $1$,
then we also halt and output 0, otherwise, if the simulation
never halts, then our program also never halts.


\begin{exer}
We say that a function $f:{\mathbb{N}}\rightarrow {\mathbb{N}}$ is
{\em nondecreasing} if
$$    x\leq y \Rightarrow f(x)\leq f(y), \mbox{ for all $x,y\in \mathbb{N}$} $$
Prove that a set $A\subseteq \mathbb{N}$ is recursive iff $A=\varnothing$
or $A$ is the range of some total computable unary nondecreasing
function $f$.  Give a careful proof, without using Church's thesis.
{\bf Hint:} For the $\Longleftarrow$ direction,
consider separately the case in which $A$ is finite.
\end{exer}


{\bf Definition:}  If $f(\vec{x})$ is a (partial) function, then
graph$(f)$ is the relation
$$   R_f(\vec{x},y) = (y=f(\vec{x}))  $$
If $f$ is a total computable function, then graph$(f)$ is a recursive
relation, since in general the substitution of a total computable
function into a recursive relation (in this case the relation $(y=z)$
is always a recursive relation (by the second Proposition,
page \pageref{second_prop}).  
However, if $f$ is computable but
not total, then graph$(f)$ is not necessarily recursive.  As an 
example, let
$f(x)=0$ if program $\{x\}$ halts on input $x$, and otherwise
$f(x)$ is undefined.   Thus  
$$  x\in K \Leftrightarrow (x,0)\in \mbox{graph}(f)  $$
Thus graph$(f)$ is not recursive, since otherwise $K$ would be recursive.

Although graph$(f)$ is not always recursive for computable functions $f$,
it is r.e.  In fact, there is a converse:

{\bf Theorem:}  Suppose $f$ is a (partial) $n$-ary function.  Then
$f$ is computable iff graph$(f)$ is recursively enumerable.

\begin{exer}
Prove the above theorem.  
\end{exer}
%To show the if direction, first give an
%informal algorithm for computing $f$ from an enumeration of the
%tuples in graph$(f)$.  Then formalize the argument by showing that
%$f$ is recursive, using the least number operator $\mu$.
%\end{exer}

{\bf Theorem} $A$ is recursive iff both $A$ and $A^c$ are r.e.

{\bf Proof:} $\Rightarrow$: Recursive sets are r.e.,
and complements of recursive sets are recursive.

$\Leftarrow$: Assume $A$ and $A^c$ are both r.e.  Then there are
recursive relations $R$ and $S$ such that
$x \in A$ iff $\exists y R(x,y)$ and
$x \in A^c$ iff $\exists y S(x,y)$.

Here is a decision procedure to determine whether $x \in A$:

\newpage

\begin{tabbing}
 \= for $y : 0 \cdots \infty$\\
 \> $\quad$ If $R(x,y)$ then output yes $(x \in A)$ {\bf exit}\\
 \> $\quad$ {\bf end if}\\
 \> $\quad$ If $S(x,y)$ then output no $(x \not\in A)$ {\bf exit}\\
 \> $\quad$ {\bf end if}\\
 \> {\bf end for}
\end{tabbing}

We know this terminates.


{\bf Application:}  $K$ is r.e. but not recursive.  Therefore by the
above theorem, $K^c$ is {\em not} r.e.

{\bf Proposition}:
Suppose $A,B\subseteq \NN$.
If $A \leq_m B$ and $B$ is r.e.\ then $A$ is r.e.

\begin{exer}
Prove the above proposition.
\end{exer}


%Use ii) in the theorem on page \pageref{reChar}
%characterizing r.e. sets.
%Since $B$ is r.e., $B=\mbox{dom}(f)$ for some computable $f$.
%Since $A\leq_m B$, there is a total computable $g$ such that
%$$  x\in A \Leftrightarrow g(x)\in B \Leftrightarrow f(g(x)) \not= \infty  $$
%then $A = \mbox{dom}(f\circ g)$.  Thus $A$ is r.e.  $\Box$

{\bf Application:} To show $A$ is not r.e., it suffices to show $K^c \leq_m A$.

\begin{exer}
Show that the following sets are not r.e.  Note that
$$   A\leq_m B \Leftrightarrow A^c\leq_m B^c  $$

$A_1=\{x \mid \{x\}_1(5) = \infty\}$\\
$A_2=\{x \mid \mbox{ran}(\{x\}_1)= {\Bbb N}\}$\\
$A_3=\{x \mid \mbox{dom}(\{x\}_1)\mbox{ is finite}\}$

Also show that $A_2^c$ and $A_3^c$ are not r.e.  In fact, it is
easier to show $A_2^c$ is not r.e. than to show $A_2$ is not r.e.
To show $A_2$ and $A_3^c$ are not r.e. use
the method suggested above (reduce $K^c$ to them).
%but use the KNFT in an interesting way.
\end{exer}

{\bf r.e. completeness:}  We say that a set $A\subseteq \mathbb{N}$
is {\em r.e. complete} iff

(i)  $A$ is r.e., and\\
(ii) for every set $B\subseteq \mathbb{N}$, if $B$ is r.e. then
$B\le_m A$.

The notion of NP-completeness was taken from the above definition.

It turns out  that every ``natural'' r.e. set $A\subseteq \mathbb{N}$
that has been shown to be not recursive is in fact r.e. complete.

\begin{exer}
Show that $K$ is r.e. complete.
\end{exer}


\bigskip
{\bf Undecidable combinatorial problems}

So far all of our examples of nonrecursive sets have referred directly
or indirectly to programs, as for example the set $K$.  However there
are many known nonrecursive sets which arrive from combinatorial
problems which on the surface appear to have nothing to do with
computation.  An example is the set $TG$ of all context-free grammars $G$
over some alphabet $\Sigma$ such that $L(G) = \Sigma^*$.  (Technically
$TG$ consists of all numerical codes for such grammars $G$, 
where we  assign a numerical code to a grammar in the same way
as we assigned codes to RM programs.)  The method for proving that
$TG$ is nonrecursive is the same as for examples above; namely
reduce $K^c$ to $TG$.  See for example ``Elements of the Theory
of Computation'' by H. R. Lewis and C. H. Papadimitriou
or ``Formal Languages and their Relation to Automata'' by
J. E. Hopcroft and J. D. Ullman for this
and other examples.

The crowning achievement for showing sets are not recursive is the
following.

\bigskip
\noindent
{\bf Hilbert's 10th Problem} (posed 1900, solved 1970)

Hilbert's problem:  Find
a procedure to determine whether a Diophantine equation
$p(\vec{x}) = q(\vec{x})$ has a solution in ${\Bbb N}$.

{\bf Definition:}  A {\em Diophantine equation} is one of the form
$p(\vec{x}) = q(\vec{x})$, where $p$ and $q$ are multivariate polynomials
with natural number coefficients.

Examples are $3x^3yz^5 + 2y^4 + 5 = 0$, and $(x+1)^n + (y+1)^n = z^n$,
for any fixed positive integer $n$.

{\bf Definition:} A {\em Diophantine} relation $R(\vec{x})$ is one of the form
$$ \exists y_1 \cdots \exists y_m\ (p(\vec{x}, y, \ldots, y_m) =
q(\vec{x}, y_1, \ldots, y_m))$$
where $p$ and $q$ are polynomials as above.

MRDP {\bf Theorem} (1970)
Every r.e.\ set is Diophantine.

{\bf Corollary}: There is no algorithm for Hilbert's 10th problem.

{\bf Proof of Corollary:} Choose any set, say $K$, which is r.e. but
not recursive.   Since $K$ is r.e., it follows from the MRDP Theorem
that $K$ has a representation of the form
$$a \in K \Leftrightarrow \exists y_1 \cdots \exists y_m (p(a, y_1 \cdots y_m)=
q(a,y_1 \cdots y_m))$$

If there were an algorithm for Hilbert's 10th, then we could determine
membership in $K$.   $\Box$

The proof of the MRDP Theorem is beyond the scope of this course.
For a readable proof, see ``Proof of recursive unsolvability
of Hilbert's Tenth Problem'' by Jones and Matiyasevich,
Amer. Math. Monthly vol. 98 (1991) 689-709.


\end{document}
