> =g(
H/0|DTimes New Roman\5D,0DWo
0DDTahomaew Roman\5D,0DWo
0D" DMonotype Sorts\5D,0DWo
00DLucida Sans Unicode,0DWo
0D"@DSymbolSans Unicode,0DWo
0Da.
@n?" dd@ @@``pb
!"#&' !()*+,-23456789:;<>=?@ABCDEFGHILMJKNO0QRTSUVWXYZ[\]^./01$%_#`a(bc$ f3f33f̙3@
g4FdFd08,ppp@uʚ;2Nʚ;<4!d!ddЁ
0P<4dddddЁ
0P<4BdBddЁ
0P___PPT9d/0HLD4,%@'(,)?O=VUndecidabilityvEverything is an Integer
Countable and Uncountable Sets
Turing Machines
Recursive and Recursively Enumerable Languages #Integers, Strings, and Other ThingsData types have become very important as a programming tool.
But at another level, there is only one type, which you may think of as integers or strings.
Key point: Strings that are programs are just another way to think about the same one data type.& ̙X
Example: Text33Strings of ASCII or Unicode characters can be thought of as binary strings, with 8 or 16 bits/character.
Binary strings can be thought of as integers.
It makes sense to talk about the i-th string. Binary Strings to Integers>There s a small glitch:
If you think simply of binary integers, then strings like 101, 0101, 00101,& all appear to be the fifth string.
Fix by prepending a 1 to the string before converting to an integer.
Thus, 101, 0101, and 00101 are the 13th, 21st, and 37th strings, respectively.rGOrG%
Example: Images33 NRepresent an image in (say) GIF.
The GIF file is an ASCII string.
Convert string to binary.
Convert binary string to integer.
Now we have a notion of the i-th image. Example: Proofs33 A formal proof is a sequence of logical expressions, each of which follows from the ones before it.
Encode mathematical expressions of any kind in Unicode.
Convert expression to a binary string and then an integer.Proofs (2)But a proof is a sequence of expressions, so we need a way to separate them.
Also, we need to indicate which expressions are given.Proofs (3)"Quick-and-dirty way to introduce new symbols into binary strings:
Given a binary string, precede each bit by 0.
Example: 101 becomes 010001.
Use strings of two or more 1 s as the special symbols.
Example: 111 = the following expression is given ; 11 = end of expression. B.uu7uNuB.33733GExample: Encoding Proofs33 Example: Programs33Programs are just another kind of data.
Represent a program in ASCII.
Convert to a binary string, then to an integer.
Thus, it makes sense to talk about the i-th program.
Hmm& There aren t all that many programs.)̙3
Finite Sets
Intuitively, a finite set is a set for which there is a particular integer that is the count of the number of members.
Example: {a, b, c} is a finite set; its cardinality is 3.
It is impossible to find a 1-1 mapping between a finite set and a proper subset of itself.R
f_33!fc
Infinite SetsZFormally, an infinite set is a set for which there is a 1-1 correspondence between itself and a proper subset of itself.
Example: the positive integers {1, 2, 3,& } is an infinite set.
There is a 1-1 correspondence 1<->2, 2<->4, 3<->6,& between this set and a proper subset (the set of even integers).Pu
fa338uCountable SetsA countable set is a set with a 1-1 correspondence with the positive integers.
Hence, all countable sets are infinite.
Example: All integers.
0<->1; -i <-> 2i; +i <-> 2i+1.
Thus, order is 0, -1, 1, -2, 2, -3, 3,&
Examples: set of binary strings, set of Java programs.P(G7
fA(33G33/!
Example: Pairs of Integers33&Order the pairs of positive integers first by sum, then by first component:
[1,1], [2,1], [1,2], [3,1], [2,2], [1,3], [4,1], [3,2],& , [1,4], [5,1],&
Interesting exercise: figure out the function f(i,j) such that the pair [i,j] corresponds to the integer f(i,j) in this order.&3fk)EnumerationsAn enumeration of a set is a 1-1 correspondence between the set and the positive integers.
Thus, we have seen enumerations for strings, programs, proofs, and pairs of integers.(f"How Many Languages?Are the languages over {0,1}* countable?
No; here s a proof.
Suppose we could enumerate all languages over {0,1}* and talk about the i-th language.
Consider the language L = { w | w is the i-th binary string and w is not in the i-th language}.&63f>6$d/"Proof Continued3f
f0Diagonalization Pictureg1Diagonalization Picture&"Proof Concluded3f
We have a contradiction: x is neither in L nor not in L, so our sole assumption (that there was an enumeration of the languages) is wrong.
Comment: This is really bad; there are more languages than programs.
E.g., there are languages with no membership algorithm.&3v+Hungarian ArgumentsWe have shown the existence of a language with no algorithm to test for membership, but we have no way to exhibit a particular language with that property.
A proof by counting the things that fail and claiming they are fewer than all things is called a Hungarian argument.(f,Turing-Machine TheoryThe purpose of the theory of Turing machines is to prove that certain specific languages have no algorithm.
Start with a language about Turing machines themselves.
Reductions are used to prove more common questions undecidable./Picture of a Turing Machine`-Why Turing Machines?Why not deal with C programs or something like that?
Answer: You can, but it is easier to prove things about TM s, because they are so simple.
And yet they are as powerful as any computer.
More so, in fact, since they have infinite memory.N.35̙T.3a.6Then Why Not Finite-State Machines to Model Computers?In principle, you could, but it is not instructive.
Programming models don t build in a limit on memory.
In practice, you can go to Fry s and buy another disk.
But finite automata vital at the chip level (model-checking).0Turing-Machine FormalismA TM is described by:
A finite set of states (Q, typically).
An input alphabet (, typically).
A tape alphabet (, typically; contains ).
A transition function (, typically).
A start state (q0, in Q, typically).
A blank symbol (B, in - , typically).
All tape except for the input is blank initially.
A set of final states (F " Q, typically).u2u+uff
ffff 2 f1Conventionsa, b, & are input symbols.
& , X, Y, Z are tape symbols.
& , w, x, y, z are strings of input symbols.
a, b,& are strings of tape symbols.6d 2The Transition FunctionpTakes two arguments:
A state, in Q.
A tape symbol in .
(q, Z) is either undefined or a triple of the form (p, Y, D).
p is a state.
Y is the new tape symbol.
D is a direction, L or R.#u?B >/ f
3Actions of the PDAIf (q, Z) = (p, Y, D) then, in state q, scanning Z under its tape head, the TM:
Changes the state to p.
Replaces Z by Y on the tape.
Moves the head one square in direction D.
D = L: move left; D = R; move right.XQ_u%uM_%9Example: Turing Machine33This TM scans its input right, looking for a 1.
If it finds one, it changes it to a 0, goes to final state f, and halts.
If it reaches a blank, it changes it to a 1 and moves left.;:Example: Turing Machine (2)33"States = {q (start), f (final)}.
Input symbols = {0, 1}.
Tape symbols = {0, 1, B}.
(q, 0) = (q, 0, R).
(q, 1) = (f, 0, R).
(q, B) = (q, 1, L).HS<Simulation of TM?Simulation of TMASimulation of TMCSimulation of TMDSimulation of TME Simulation of TMK#.Instantaneous Descriptions of a Turing MachineInitially, a TM has a tape consisting of a string of input symbols surrounded by an infinity of blanks in both directions.
The TM is in the start state, and the head is at the leftmost input symbol.I!TM ID s (2)nAn ID is a string aqb, where ab is the tape between the leftmost and rightmost nonblanks (inclusive).
The state q is immediately to the left of the tape symbol scanned.
If q is at the right end, it is scanning B.
If q is scanning a B at the left end, then consecutive B s at and to the right of q are part of a.rc`O L$TM ID s (3)As for PDA s we may use symbols " and "* to represent becomes in one move and becomes in zero or more moves, respectively, on ID s.
Example: The moves of the previous TM are q00"0q0"00q"0q01"00q1"000f a33&O%Formal Definition of Moves,If (q, Z) = (p, Y, R), then
aqZb"aYpb
If Z is the blank B, then also aq"aYp
If (q, Z) = (p, Y, L), then
For any X, aXqZb"apXYb
In addition, qZb"pBYbu0uu-u >vR&Languages of a TMA TM defines a language by final state, as usual.
L(M) = {w | q0w"*I, where I is an ID with a final state}.
Or, a TM can accept a language by halting.
H(M) = {w | q0w"*I, and there is no move possible from ID I}.Z?b.S'$Equivalence of Accepting and HaltingIf L = L(M), then there is a TM M such that L = H(M ).
If L = H(M), then there is a TM M such that L = L(M ).pupT(!Proof of 1: Acceptance -> Halting"3fModify M to become M as follows:
For each accepting state of M, remove any moves, so M halts in that state.
Avoid having M accidentally halt.
Introduce a new state s, which runs to the right forever; that is (s, X) = (s, X, R) for all symbols X.
If q is not accepting, and (q, X) is undefined, let (q, X) = (s, X, R)."ouu"oBAX)!Proof of 2: Halting -> Acceptance"3fModify M to become M as follows:
Introduce a new state f, the only accepting state of M .
f has no moves.
If (q, X) is undefined for any state q and symbol X, define it by (q, X) = (f, X, R).X"u"L?Z* Recursively Enumerable Languages4We now see that the classes of languages defined by TM s using final state and halting are the same.
This class of languages is called the recursively enumerable languages.
Why? The term actually predates the Turing machine and refers to another notion of computation of functions.<n fn[+Recursive LanguagesAn algorithm is a TM that is guaranteed to halt whether or not it accepts.
If L = L(M) for some TM M that is an algorithm, we say L is a recursive language.
Why? Again, don t ask; it is a term with a history.R5f~f5qc^,Example: Recursive Languages33Every CFL is a recursive language.
Use the CYK algorithm.
Every regular language is a CFL (think of its DFA as a PDA that ignores its stack); therefore every regular language is recursive.
Almost anything you can think of is recursive.6##/@
#
$(*-.45678:=>@BFGH J!N"P#Q$U%V&W'Y(\)]*_+b,c-e.h/i0 ` ̙33` ` ff3333f` 333MMM` f` f` 3>?" dd@,?udd@ w " @ ` n?" dd@ @@``PR @ ` `p>>f(
6p P
T Click to edit Master title style!
!
0Xs
RClick to edit Master text styles
Second level
Third level
Fourth level
Fifth level!
S
04x ``
>*
0} `
@*
0| `
@*Z
Byh@ ? ̙33 Default Design
0zrP
(
0z P
P*
0X~
R*
d
c$ ?
0{
@
RClick to edit Master text styles
Second level
Third level
Fourth level
Fifth level!
S
6 `P
P*
6 `
R*
H
0h ? ̙330$(
r
S0
r
SU`
H
0h ? ̙33
`$$(
$r
$ SP
r
$ S<
H
$0h ? ̙33
,$(
,r
, SP
r
, S|
H
,0h ? ̙33
0$(
0r
0 S|P
r
0 S$
H
00h ? ̙33
<P(
<r
< S\P
< S<<$
0
H
<0h ? ̙33
D$(
Dr
D S,P
r
D S
H
D0h ? ̙33
L$(
Lr
L SP
r
L S
H
L0h ? ̙33
T:(
Tr
T SP
T S 0
"p`PpH
T0h ? ̙33
IA@ ` (
`r
` S P
`
<8
`M
v:1110100011111100000101110101& l PG<
`PG<
,$D
0
`
<PPG<
JA given
expression
follows`
`
0`0ZB
`
s*D0Pl p
` p,$D
0
`
<
S
aAn ex-
pression`
`
0p0ZB
`
s*D00@l pV
`pV ,$D
0
`
<PV
AEnd of
expression`
`
0p 0ZB
`
s*D 0 Pl r
`r,$D
0
`
<`pr
XNotice this
1 could not
be part of
the end --ZB
`
s*D0 l
` ,$D
0f
`
6 @0
`
B
JA given
expression
follows`B
`
0DP
0 l @~p
`@~p,$D
0f
`
6@0
`
B
P~p
:
Expression`B
`
0D0@ l
` ,$D
0
`
<H$
3End`
`
00ZB
`
s*DP0`H
`0h ? ̙33
`hP(
hr
h S/P
h S0<$
0
H
h0h ? ̙33
p$(
pr
p S<P
r
p S<
H
p0h ? ̙33
t$(
tr
t SJ
r
t SK `0
H
t0h ? ̙33
$(
r
SlYP
r
S(Z`
H
0h ? ̙33
$(
r
SjP
r
SHk
H
0h ? ̙33
$(
r
SuP
r
Su
H
0h ? ̙33
P(
r
S
SP0P<$
0
H
0h ? ̙33[
.@
(
r
S4P
0
Clearly, L is a language over {0,1}*.
Thus, it is the j-th language for some particular j.
Let x be the j-th string.
Is x in L?
If so, x is not in L by definition of L.
If not, then x is in L by definition of L.F uT w T,8/gc
<8@
,,$D
0
QRecall: L = { w | w is the
i-th binary string and w is
not in the i-th language}."R3L,%Uz 0K
0K ,$D
0
<@
1xZB
B
s*D[ZB
B
s*Dp[ZB
s*D0K z
5
,$D
0
<
Vj-thZB
s*D
ZB
s*D0+z 0P
0P
,$D
0
<0P
XLj ZB
B
s*DP@ZB
s*D P
H
0h ? ̙33
/4,""(
r
SP
LB
c$DLB
c$D@LB
c$D` ` @LB
c$D
@LB
c$D``@LB
c$D
@LB
c$D``@LB
c$D@@LB
c$D LB
c$D
LB
c$D00LB
c$D
<e q
7Strings
<
f61 2 3 4 5 &
<NV3%
11
<XK@
`
11
</@`=`
12
<P `=
13
<HV
`=
14
<\``=
15
<6
0A
2&
<T~`
9 Languages
<$
10
<4
11
<@
11
<@
11
<00@P
11
< @
10
<Ƹ@
10
≮
10
!
<d
2&
"
<
2& H
0h ? ̙33
0ph''(
x
c$8P
LB
c$DLB
c$D@LB
c$D` ` @LB
c$D
@LB
c$D``@LB
c$D
@LB
c$D``@LB
c$D@@LB
c$D LB
c$D
LB
c$D00LB
c$D
<2e q
7Strings
<LT
f61 2 3 4 5 &
<TV3%
11
<X@
`
50f
<@`=`
12
<ʋ `=
13
<L
`=
14
<]``=
15
<,_
0A
2&
<(`
9 Languages
<K
51f
<\M
11
<Ȟ@
11
<@
50f
<0@P
50f
< @
51f
<{@
10
<8
10
!
<D
2&
"
<M
2&
#
<@@,
HFlip each
diagonal
entry^l P
'P,$D
0f2
$
6<' @
%
<`
pP
nCan t be
a row
it disagrees
in an entry
of each row.88ZB
&B
s*D
@
H
0h ? ̙33
`$(
r
SP
r
S|
H
0h ? ̙33
$(
r
SP
r
Sp
H
0h ? ̙33
P(
r
S$͈P
S͈<$
0
H
0h ? ̙33
1(
r
S؈P
<ڈ̙ 0
7StateLB
c$D@@LB
c$DLB
c$D@P
P
LB
c$D@LB
c$D@LB
c$D@@@LB
c$D@ LB
c$D@
<L%VE
5. . .
<`0
5. . .
<p@`
1A
<P@0
`
1B
<8@
g`
1C
<@`
1A
<@
`
1DRB
s*D
@
<(-
PInfinite tape with
squares containing
tape symbols chosen
from a finite alphabetQQ[
<$
,$D
0
}Action: based on
the state and the
tape symbol under
the head: change
state, rewrite the
symbol and move the
head one square."~3fxH
0h ? ̙33
,P(
r
SDP
S$ `<$
0
H
0h ? ̙33
-P(
r
SdP
SD<$
0
H
0h ? ̙33
F(
x
c$\
c$$0
"p`PpH
0h ? ̙33
@0(
x
c$>P
x
c$>
H
0h ? ̙33
`F(
x
c$\GP
c$t`
"p`PpH
0h ? ̙33
F(
x
c$ XP
c$0
"p`PpH
0h ? ̙33
$(
r
SiP
r
STj
H
0h ? ̙33
$(
r
S__o}ql r[J9!tvx+rz)-NF|/&~135v7J9;*=ҌxzأN;"֗iOh+'00`h
CS154 slidesoJeff UllmanJeffUll81fMicrosoft PowerPointP@Pc(dB@}J@0JI/G/g 4& &&#TNPP2OMi
&
TNPP &&TNPP
&&--&&-$0>$>L$LZ*$*Zh8$8hvF-$FvT$Tb$bp-$p~$~$-$-$$-$$-$-$,$,:
-$
:H$HV&-$&Vd4-$4drB-$BrP$P^-$^k-$ky-$y-$-$-$-$-$-$-$-$'-$'5-$5C-$CQ!-$!Q_/-$/_m=-$=m{K-$K{Y-$Yg-$gu-$u-$-$-$-$-$-$-$-$#-$#1-$1?-$?M-$M[+-$+[i9-$9iwG-$GwU-$Uc-$cq-$q-$-$-$-$-$-$-$-$-$--$-;-$;I-$IW'-$'We5-$5erB-$BrP-$P^-$^l-$lz-$z-$-$-$-$-$$-$-$($(6-$6D-$DR"$"R`0-$0`n>$>n|L-$L|Z$Zh$hv-$v$$$&&&-&$&&-&&
&&-&&&&-$0>$>L$LZ*$*Zh8$8hvF-$FvT$Tb$bp-$p~$~$-$-$$-$$-$-$,$,:
-$
:H$HV&-$&Vd4-$4drB-$BrP$P^-$^k-$ky-$y-$-$-$-$-$-$-$-$'-$'5-$5C-$CQ!-$!Q_/-$/_m=-$=m{K-$K{Y-$Yg-$gu-$u-$-$-$-$-$-$-$-$#-$#1-$1?-$?M-$M[+-$+[i9-$9iwG-$GwU-$Uc-$cq-$q-$-$-$-$-$-$-$-$-$--$-;-$;I-$IW'-$'We5-$5erB-$BrP-$P^-$^l-$lz-$z-$-$-$-$-$$-$-$($(6-$6D-$DR"$"R`0-$0`n>$>n|L-$L|Z$Zh$hv-$v$$$&-
--&&&y&/xv`{v
m8C.xv`{vfg- @Times New Roman\xv`{vfg- . 2
f1 .--q@-- @"Tahomav
p8C.\xv`{vfg- .2
$Undecidability'!
!!
.--1 -- @"Tahomav
C.\xv`{vfg- .+2
OEverything is an Integer
. .42
Countable and Uncountable Sets
. .2
ETuring Machines
!
. ..2
Recursive and Recursively
. .%2
:Enumerable Languages$
.--"Systemgfg
G-&TNPP &՜.+,0
(On-screen ShowStanford University, CS Dept.LA00 6Times New RomanTahomaMonotype SortsLucida Sans UnicodeSymbolDefault DesignUndecidability$Integers, Strings, and Other ThingsExample: TextBinary Strings to IntegersExample: ImagesExample: Proofs
Proofs (2)
Proofs (3)Example: Encoding ProofsExample: ProgramsFinite SetsInfinite SetsCountable SetsExample: Pairs of Integers
EnumerationsHow Many Languages?Proof ContinuedDiagonalization PictureDiagonalization PictureProof ConcludedHungarian ArgumentsTuring-Machine TheoryPicture of a Turing MachineWhy Turing Machines?7Then Why Not Finite-State Machines to Model Computers?Turing-Machine FormalismConventionsThe Transition FunctionActions of the PDAExample: Turing MachineExample: Turing Machine (2)Simulation of TMSimulation of TMSimulation of TMSimulation of TMSimulation of TMSimulation of TM/Instantaneous Descriptions of a Turing MachineTM IDs (2)TM IDs (3)Formal Definition of MovesLanguages of a TM%Equivalence of Accepting and Halting"Proof of 1: Acceptance -> Halting"Proof of 2: Halting -> Acceptance!Recursively Enumerable LanguagesRecursive LanguagesExample: Recursive LanguagesFonts UsedDesign Template
Slide Titles0_゙JeffJeff
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~Root EntrydO)Current UserSummaryInformation(0PowerPoint Document(DocumentSummaryInformation8Root EntrydO)\Current UserPSummaryInformation(0PowerPoint Document((_゙_Michael GoodrichMichael Goodrich__