Spectral partitioning

Statement and Proof of Theorem 4.

(CS 267, Mar 23 1995)

Theorem 3. (Special case of the Courant Fischer Minimax Theorem). If A is a symmetric matrix with eigenvalues lambda(1) <= lambda(2) <= ... and eigenvectors v(1), v(2), ..., then

       lambda(2) = min_{v != 0,  v'*v(1) = 0}  v'*A*v / v'v

and the minimizing v is v(2).

Proof. Substitute the eigendecomposition A = Q*Lambda*Q', where Q is a orthogonal matrix whose columns v(i) are eigenvectors, and Lambda = diag(lambda(1), ..., lambda(n)) is a diagonal matrix of eigenvalues, into the expression in the theorem:

       min_{v != 0,  v'*v(1) = 0}  v'*A*v / v'v
           = min_{v != 0,  v'*v(1) = 0}  v'*Q*Lambda*Q'*v / v'*v
           = min_{Qv != 0,  v'*Q*Q'*v(1) = 0}  v'*Q*Lambda*Q'*v / v'*Q*Q'*v
           = min_{y != 0,  y'*y(1) = 0}  y'*Lambda*y / y'*y
                     where y = Q'*v and y(1) = Q'*v(1) = [1,0,...,0]'
           = min_{y != 0, y_1 = 0} sum_i lambda(i)*y_i^2 / sum_i y_i^2

It is easy to see that this expression is minimized by taking y = [0,1,0,...,0]', yielding lambda(2). Then v = Q*y = v(2), as desired. QED