CS70 - Lecture 6 - Jan 31, 2011 - 10 Evans
We are increasing sections sizes, waitlist down to 2.
Section 105 (T 3-4) still has 4 slots.
Our next goal is to use induction to analyze algorithms.
In other classes you learned about recursion.
Here our point is that recursion and induction are two
sides of the same coin: recursion is how an algorithm
works and induction is how we analyze it to either prove
it is correct or figure out how long it takes to execute.
EG: Fibonacci numbers: Let F(0)=0, F(1)=1 and F(n)=F(n-1)+F(n-2).
So F(0,1,2,…) = 0,1,1,2,3,5,8,13,21,...
Theorem: Let x_+ = (1+sqrt(5))/2 ~ 1.6 and x_- = (1-sqrt(5))/2 ~ -.6
Then F(n) = ( x_+^n - x_-^n )/sqrt(5) ~ 1.6^n / sqrt(5)
grows exponentially fast
Proof by induction:
P(n) = " F(n) = ( x_+^n - x_-^n )/sqrt(5)"
Bases case(s): Check P(0) and P(1) are true,
i.e. that the formula yields F(0) = 0 and F(1) = 1 as desired
Induction step: show that P(n-2) and P(n-1) -> P(n):
P(n-2) and P(n-1) ->
F(n-2)+F(n-1) = ( x_+^(n-2) - x_-^(n-2) )/sqrt(5) + (x_+^(n-1) - x_-^(n-1))/sqrt(5)
= … = ( x_+^n - x_-^n)/sqrt(5)
= F(n) -> P(n)
Consider following 2 algorithms for computing F(n):
if n=0 return 0
elseif n = 1 return 1
x = 0, y = 1,
for i= 2 to n
tmp = y, y = x+y, x=tmp
(Note: bug in class notes; what does the function there compute instead?)
if n=0 return 0
else if n= 1 return 1
else return F2(n-1) + F2(n-2)
Which algorithm is faster? More simply: how many additions does each one perform?
Let A1(n) = #additions_in_F1(n) = ?
Let A2(n) = #additions_in_F2(n) = 1 + A2(n-1) + A2(n-2)
So A2(0,1,2,…) = 0, 0, 1, 2, 4, 7, 12, 20,...
What is the relationship between A2(n) and F(n)?
Looks like A2(n) = F(n+1)-1.
Proof by induction: A2(n) = 1 + A2(n-1) + A2(n-2)
= 1 + (F(n)-1) + (F(n-1)-1) … by induction
= F(n) + F(n-1) -1
= F(n+1) -1 … as desired!
Looks like F1(n) is *much* faster than F2(n)
Ex: Assume your compute takes 1 nanosecond to add two numbers.
How long does it take to evaluate F1(129)? About 128 nanoseconds
How long does it take to evaluate F2(129)? About
F(130) nanoseconds ~ 10^27 nanoseconds ~ 34 billion years
How old is the universe?
Now suppose you had one computer for each atom in the universe,
about 10^80 of them, all running in parallel to help you run program F2.
How long would it take to compute F2(512)? About
F(513) / 10^80 nanoseconds ~ 10^107/1e^80 nanoseconds ~ 23 billion years
How would you ever guess the formula for F(n)? Exactly: guess!
Try F(n) = x^n and see what x has to be for this to be true:
F(n) = x^n = F(n-1) + F(n-2) = x^(n-1) + x^(n-2)
or x^n = x^(n-1) + x^(n-2)
or x^2 = x + 1
or x = (1+sqrt(5))/2 = x_+ or x = (1-sqrt(5))/2 = x_-
So both F(n) = x_+^n and F(n) = x_-^n satisfy F(n) = F(n-1)+F(n-2).
But neither satisfies F(0)=0 and F(1) = 1: what to do?
Note that for any constants r and s, F(n) = r*x_+^n + s*x_-^n
also satisfies F(n) = F(n-1) + F(n-2), so we can pick the 2 constants r and s
to satisfy the 2 constraints F(0)=0 and F(1) = 1 (or F(0)=7 and F(1) = -pi, whatever we like).
The same "guessing" procedure works for similar recurrences, like
G(n) = 2*G(n-1) - 7*G(n-2), G(0) = 2, G(1) = -3
Try plugging in G(n) = x^n, solve a quadratic for two values of x, etc.
What do you think happens with
H(n) = 3*H(n-1) + 2*H(n-2) - H(n-3)?
Such "linear recurrences" occur commonly, eg in analyzing signal processing.
Next example of using induction to analyze and algorithm, this time for sorting.
One of the fastest algorithms is called quicksort, and it works like this.
function quicksort(n, A)
… input is array A of n numbers
… output is array of these n numbers sorted in increasing order
if (n=0 or n=1)
pick a random number 1 <= i <= n
reorder the entries of A so that
the initial entries are all <= A(i) (say there are m of them)
the next entry = A(i)
the remaining entries are > A(i)
return S = [quicksort(m,A),A(i),quicksort(n-m-1,A(m+1:n))]
The function quicksort is recursive, and we will prove it correctly
sorts by using induction on the length of the array being sorted
P(n) = "quicksort correctly sorts an input array of length n"
Base cases: P(0) and P(1) work because the algorithm doesn't have to do anything
Induction step: We assume P(0) and P(1) and … and P(n) and prove P(n+1):
after reordering array A of length n+1, it is partitioned into 3 subsets:
(1) entries <= A(i), except A(i) itself
(3) entries > A(i)
Obviously, if we correctly sort subsets (1) and (3), the whole array will be sorted.
Quicksort correctly sorts these subsets because their lengths are at most n,
since they don't contain A(i).