------------------------
                Math.464   PROBLEM SET 6
                ------------------------
Problems on the Psudoinverse and related things.

1. Preliminaries.
-------------
Given a matrix A(mXn) of rank r<= min(m,n), we can factor as
 A = BC with B(mXr) and C(rXn), both of rank r. A simple way to
see that for any matrix A is by using the LU factorization.
Since for P a permutation we have PA = LU with U in upper echelon form,
we have that U will be of form:
                  n
U = /*.............\ =: / C \    rxn
    |0*............|    \ 0 /  (m-r)Xn
    |  0*..........|
    |   0  *..     |r
    |     0       0|
    \      0       /m
Then, since
                   m
L = /10             \
    |.10            |
    |..10           |
    |...            |
    \.. .. ..      1/ m
we easily see that the last (m-r) columns do not enter in computing
PA = LU
so we can replace the last m-r columns of L by zeros:
PA = L'U, with 
L' = < B' 0 >, B' mXr.
so that PA = LU = B'C where B' is mXr and C is rXn, both of rank r
(B' is made from the first r columns of L which are independent (L nonsingular)
and C is made frm the top r rows of U which are also independent as the
pivotal rows). Finally
 PA = B'C => A = (PB')C = BC as claimed.
Here B is made from the first r columns of L, as permuted by P.
It is easy to verify that R(B) = R(A) and R(C^T) = R(A^T) (be sure you
can see that!).
 
 Now both B^T B and CC^T are nonsingular and so their inverses exist
and we can define Ci = (CC^T)^(-1), Bi = (B^T B)^(-1) and the pseudoinverse
A+ of A is defined as
                       A+ := C^T Ci Bi B^T. 
Although we showed that the solution of the normal equations
A^T A x = A^T b is given by
x = (A+)b
you get its proof as a bonus problem below!.

 We also Showed that (assuming A is already written in such a way that its
first r columns (a_1,..,a_r) give a basis of its column space; in the proof
above we did not assume that!) A can be written in the equivalent forms:

A = /A11  A12\
    \A21  A22/ with A11(rXr) and the other submatrices dimensioned
accordingly, or
A =/I_r\ <A11 A12> = /A11\ <I_r Q>
   \ P /             \A21/
for some P((m-r)Xr) and Q(rX(n-r)) and I_r the rXr identity.
 The problems below are for the A, B, C etc. as defined above.

----------------------------------------
2. Supplementary problems for homework 6
----------------------------------------
                                             
<6.1> SHow that A can be written as
     A =/I_r\ A11 <I_r Q>
        \ P /
<6.2> Show that
 (a) N(C) = N(A), i.e. Ch = 0 <=> Ah = 0
 (b) A(A+)A = A ; (A+)A(A+) = A+
 (c) (AA+)^T = AA+ and  ((A+)A)^T = (A+)A

<6.3> Show that 
 (a) x=(A+)b solves the normal equs. A^T A x = A^T b
 (b) if xER(A^T) then (A+)A x = x
 (c) if xEN(A)   then (A+)A x = 0
 (d) if yEN(A^T) then (A+)  y = 0
and, finally
 (e) if A x = y, with xER(A^T) (and yER(A))
then  (A+)y = x .
(Note: E means "belongs" )
----------------------------------------
The rest of the problems are as given:
    Set 6 (due 10/4):p. 180, 3.4(2,4,5,6,9,13,15,18,25)
                     p. 205, 3.6(8,17,20,22)