CS70 - Lecture 10 - Feb 9, 2011 - 10 Evans

Goal for Note 5:

Modular arithmetic

How computers do integer arithmetic

computing gcds

Theorem (division algorithm) given integers a, d>0 (divisor), there is a

unique q (quotient) and r (remainder) such that 0<=r<d, a=q*d+r

DEF if a and d>0 are integers as above, then a mod d = r,

remainder after dividing a by d

Application of Division Algorithm: Modular Arithmetic

DEF We say "a is congruent to b mod d" (or "a == b mod d") if d|(a-b)

Otherwise we say "a !== b mod d"

EX: 3 == 17 mod 7 because 7 | (17-3)

Thm 1: a == b mod d if and only if there is an integer k such that

a = b + k*d

proof (=>) a == b mod d -> d|(a-b) -> exists k: a-b=k*d -> a=b+k*d

proof (<=) a=b+k*d -> a-b = k*d -> d|(a-b) -> a == b mod d

EX: 3 == 17 mod 7 <=> 7 | (17-3)  <=>  17=3+2*7

Thm 2: a == b mod d if and only if a mod d = b mod d:

proof (<=) a mod d = b mod d -> a=qa*d+r, b=qb*d+r, ->

(a-b)=(qa-qb)*d, -> d|(a-b) -> a == b mod d

proof (=>) a == b mod d -> a = b + k*d by Thm 1, so when dividing

a = qa*d+ra, b=qb*d+rb, with 0 <= ra,rb < d, we get

ra-rb = (a-qa*d)-(b-qb*d) = (a-b)+qb*d-qa*d = (k+qb-qa)*d

now -d < ra-rb < d and ra-rb is also a multiple of d,

so ra-rb=0

EX: 3 == 17 mod 7 <=> 3 mod 7 = 17 mod 7

ASK&WAIT: is 111 == 63 mod 3? is 123 == 6789 mod 2?

Thm 3: a==b mod d and c==e mod d => a+c == b+e mod d

proof: a = qa*d+r1 and b = qb*d+r1 and c = qc*d+r2 and e = qe*d+r2 ->

(a+c)=(qa+qc)*d+r1+r2 and b+e=(qb+qe)*d+r1+r2 ->

(a+c)-(b+e) = d*(qa+qc-qb-qe) -> d|(a+c-b-e) -> a+c == b+e mod d

EX: 3==17 mod 7 and 11==4 mod 7 => 3+11 == 17+4 mod 7  (7|(21-14), i.e. 7|7)

ASK&WAIT: Is 112+227 == 31+65 mod 3?

Thm 4: a==b mod d and c==e mod d => a*c == b*e mod d

(try to prove this yourself)

EX: 3==17 mod 7 and 11==4 mod 7 => 3*11 == 17*4 mod 7  (7|(68-33), i.e. 7|35)

ASK&WAIT: Is 112*227 == 31*65 mod 3?

DEF: "arithmetic modulo d" or "modular arithmetic with modulus d" means

doing integer arithmetic (+,-,*) where any two a,b satisfying

a == b mod d are considered the same, because we only care what

Thm 3 means that if we want to add and subtract numbers mod d, we can

take any number or intermediate result and add a multiple of d to it

(replace it by its value mod d) without changing the final answer.

Thm 4 says the same thing about multiplication

Ex: (13+15)*(2+8) mod 8 can be computed the following equivalent ways:

(1) ((13+15)*(2+8)) mod 8 = 280 mod 8 = 0 mod 8

(2) 13+15==28 mod 8 == 4 mod 8 and 2+8==10 mod 8 ==2 mod 8 so

((13+15)*(2+8)) mod 8 == 4*2 mod 8 == 8 mod 8 == 0 mod 8

Here are several useful applications of modular arithmetic:

Thm: Let x = d(n-1)d(n-2)...d(0) be an n-digit decimal integer.

Then 3|x if and only if 3 | d(n-1)+d(n-2)+...+d(0), i.e. the

sum of x's decimal digits.

Proof. We want to show that x mod 3 = 0 if and only if

d(n-1)+...+d(0) mod 3 = 0.

x == sum_{i=0 to n-1} d(i)*10^i  mod 3 ... by def of decimal number

== sum_{i=0 to n-1} d(i)*(10^i mod 3)  mod 3

== sum_{i=0 to n-1} d(i)*(10 mod 3)^i  mod 3

== sum_{i=0 to n-1} d(i)*(1)^i  mod 3

== sum_{i=0 to n-1} d(i) mod 3

as desired

Thm: Let x = d(n-1)d(n-2)...d(0) be an n-digit decimal integer.

Then 9|x if and only if 9| d(n-1)+d(n-2)+...+d(0), i.e. the

sum of x's decimal digits.

Proof: the same as above, substituting 9 for 3

EX: Simplest way to implement "arithmetic modulo 8" means

only use numbers in set S={0,1,2,3,4,5,6,7};

after every operation (like 3*4) take result modulo 8 to get

number (4) in S

EX: Take a disk with d=8 equispaced points on circumference, labelled

0 (at top), 1 (to right) and around to d-1 = 7. Take another similar

disk with same center. to add a+b, align 0 of second disk

with a of first disk, and see where b of second disk hits

first disk, namely at a=b mod d ("circular slide rule").

Multiplication can be thought of as repeated addition.

(picture on board)

EX: Computer Implementation of arithmetic mod 8:

"Unsigned Integers" in C, C++

represent numbers in base 2 (with 3 binary digits, or bits):

to left, where you get a "carry" from one column of bits to the

next if the sum in the column is at least 2 (10 in decimal), as

indicated below. If there is a carry out of the last column, we

ignore it, since there is no place to "carry it to".

This discarded carry represents an 8, so the final sum is 8 smaller

than the true value, eg 12-8=4 instead of 12, which is the

carries: 000              100               110               000

001_2 = 1        010_2 = 2         101_2 = 5         100_2 = 4

+ 010_2 = 2      + 011_2 = 3       + 111_2 = 7       + 101_2 = 5

-----------      -----------       -----------       -----------

011_2 = 3        101_2 = 5         100_2 = 4         001_2 = 1

= 12 mod 8        = 9 mod 8

EX: 2's complement arithmetic: how computers do integer arithmetic

with positive and negative integers.

Suppose computer words had just 3 bits, representing 2^3=8 numbers.

(A real computer would use 32 bits, representing 2^32 numbers, but

it is the same idea.). Then instead of doing modular arithmetic on

the set S={0,1,2,3,4,5,6,7}, we use the set S={-4,-3,-2,-1,0,1,2,3}.

I.e. after each operation, we add a multiple of 8 (or 2^32) to the

result to get an answer in S.

The way you tell positive from negative numbers among the 8 bit

patterns on our 3-bit computer is to look at the leftmost bit,

the "sign bit": it is 0 for nonnegative numbers, and 1 for negative.

bit pattern   unsigned integer  2's complement integer

-----------   ----------------  ----------------------

000              0                   0

001              1                   1

010              2                   2

011              3                   3

100              4                  -4 = 4-8

101              5                  -3 = 5-8

110              6                  -2 = 6-8

111              7                  -1 = 7-8

Rule to interpret bit pattern as 2's complement number:

if leading bit ("sign bit") is 0

the number is positive, with the same value as the unsigned integer

else if the sign bit is 1

the number is negative, with the value of unsigned integer minus 8

There are "circular slide rules" for unsigned (draw on board) and

2s complement (draw on board) integer arithmetic:

With  3 bits, numbers range from -2^2      = 100_2    =  -4

to  2^2-1    = 011_2    =   3

With 32 bits, numbers range from -2^31     = 10...0_2 = -2147483648

to  2^31 - 1 = 011..1_2 =  2147483647

Arithmetic is done the same way, whether unsigned or 2's complement:

carries: 000              100               110               000

001_2 = 1        010_2 = 2         101_2 = -3        100_2 = -4

+ 010_2 = 2      + 011_2 = 3       + 111_2 = -1      + 101_2 = -3

-----------      -----------       -----------       -----------

011_2 = 3        101_2 = -3        100_2 = -4        001_2 = 1

ASK:  int a,b,c,d,e,f on 3 bit machine

a = 3

b = a+1                ... what is b?

c = a+2                ... what is c?

d = 2

e = 2*d                ... what is e?

f = 2*e                ... what is f?

ASK:  int a,b,c,d,e,f on 32 bit machine (like most)

a = (2^30-1) + 2^30    ... what is a?

b = a+1                ... what is b?

c = a+2                ... what is c?

d = 2^30               ... what is d?

e = 2*d                ... what is e?

f = 2*e                ... what is f?

Try running above program on your own machine

What does division mean in modular arithmetic?

With real numbers: x = 2/3 same as solving 3*x=2 for x

With modular arithmetic (mod 7, say),  we solve 3*x == 2 mod 7

solve 3*y == 1 mod 7 (if we can)

multiply by 2 to get 3*(2*y) == 2*1 == 2 mod 7, so x = 2*y

Can we always compute multiplicative inverses?

Ex:  3*x == 1 mod 7  satisfied by x=5

Ex:  12*x == 1 mod 15  satisfied by no x!

Proof: if it were, we would have 15 | (12*x-1)

or 15*d = 12*x - 1 or 15*d - 12*x = 1

or 3*(5*d - 4*x) = 1 or 3 divides 1 - contradiction!

Def: gcd(m,n) = greatest common division of m and n

= largest positive integer d such that d|m and d|n

Theorem: We can solve m*x == 1 mod n iff gcd(m,n) = 1

Proof: (=>) Proof by contrapositive: If gcd(m,n) = q > 1,

then q | m*x + y*n for any y so q | (m*x mod n),

so m*x mod n neq 1 = 1 mod n, i.e. m*x !== 1 mod n for any x.

(<=) Consider the sequence of numbers {0,m,2*m,3*m,…,(n-1)*m}.

We claim these are all distinct modulo n; we prove this

by contradiction: If they were not all distinct, that is if

i*m == j*m mod n for some 0 <= i < j < n, then n | (j-i)*m.

Since gcd(n,m) = 1, n can't have any common factors with m,

so it must divide j-i. But 0 <= j-i < n so n can't divide j-i either,

contradiction. And since they are all distinct modulo n, they

must range over all n possible values modulo n: {0,1,2,…,n-1},

so for some x, m*x == 1 mod n.

How to compute gcd quickly, using Euclidean algorithm

Computing gcd(m,n) by factoring m and n into products of primes and

taking all the common prime factors is very expensive (cryptography

depends on this!) so we want a faster algorithm:

Euclidean algorithm:

func  gcd(m,n) ….  assume m >= n >= 0

if n=0 return m else return gcd(n, m mod n))

Thm: The Euclidean algorithm computes gcd(m,n) in O(#bits in m) iterations (m is the larger argument).

Proof: d|m and d|n implies d|(m-q*n) for any q, so in particular d|(m mod n).

Conversely, d|n and d|(m-q*n) implies d|(m-q*n+q*n) or d|m. Thus

(m,n) have the same set of common divisors as (n, m mod n) and so gcd(m,n)=gcd(n, m mod n).

Furthermore, gcd(m,0) = m, so if the algorithm terminates, it must return the right answer.

To prove it must terminate, we show that the number of bits in the first (larger) argument

must decrease by at least 1 every 2 steps; this means the algorithm must stop after

at most 2*(#bits in m) steps. There are two cases:

Case 1: if n <= m/2, then n is at least one bit shorter than m after 1 step.

Case 2: if n > m/2, then the next call is to gcd(n, m mod n) = gcd(n,m-n), and the one after that

is to gcd(m-n,gcd(n,m-n)). Again, m-n < m/2 is at least one bit shorter than m.

Ex: gcd(15,12) => gcd(12,3) => gcd(3,0) = 3

gcd(15,11) => gcd(11,4) => gcd(4,3) => gcd(3,1) => gcd(1,0) = 1

How to compute multiplicative inverses quickly, using extended Euclidean algorithm:

Suppose given (m,n), we could find integers x and y so that  d = gcd(m,n) = m*x + n*y.

In particular, suppose gcd(m,n)=1, so that x and y satisfies 1 = m*x + n*y;

what would this tell us?  That m*x == 1 mod n, i.e. x is the multiplicative inverse of m.

We can extend the Euclidean algorithm to return (x,y) as follows:

func [d, x, y] = extended_gcd(m,n) … return d,x,y satisfying gcd(m,n)=d=m*x+n*y

if n = 0 return [m,1,0]

else [d, x, y] = extended_gcd(n, m mod n), return [d, y, x-floor(m/n)*y]

(Here floor(m/n) = quotient when dividing m by n

= m/n rounded down to the nearest integer.)

Thm: The extended Euclidean Algorithm returns d = gcd(m,n) and integers x,y s.t. d = m*x + n*y

Proof: The fact that the algorithm terminates with d = gcd(m,n) is the same as before.

It remains to show, by induction, that d = m*x + n*y.

The base case is when n=0, in which case x=1, y=0, and m = m*1 + 0*0 as desired.

For the induction step, suppose that d = gcd(n, m mod n) = n*x + (m mod n)*y.

Recall that m mod n = m - floor(m,n)*n. Substituting we get

d = n*x + (m - floor(m,n)*n)*y = m*y + n*(x - floor(m,n)*y)

which shows the algorithm returns the correct coefficients y and x-floor(m,n)*y.

Ex: gcd(15,12) calls gcd(12,3) calls gcd(3,0)

returns 3 = 3*1+0*0 returns 3 = 12*0 + 3*(1-floor(12/3)*0) = 12*0 + 3*1

returns 3 = 15*1 + 12*(0-floor(15/12)*1) = 15*1 + 12*(-1)

Ex: gcd(15,11) calls gcd(11,4) calls gcd(4,3) calls gcd(3,1) calls gcd(1,0)

returns 1 = 1*1 + 0*0

returns 1 = 3*0 + 1*(1-floor(3/1)*0) = 3*0 + 1*1

returns 1 = 4*1 + 3*(0-floor(4/3)*1) = 4*1 + 3*(-1)

returns 1 = 11*(-1)  + 4*(1-floor(11/4)*(-1)) = 11*(-1) + 4*3

returns 1 = 15*3  + 11*(-1-floor(15/11)*3) = 15*3 + 11*(-4)