SVD/Eigenvector confusion
On Sun, 29 Feb 2004, Philip Warner wrote:
My understanding of SVD is that, for A an mxn matrix, m > n:
A = UWV*
where W is square root diagonal eigenvalues of A*A extended with zero
valued rows, and U and V are the left & right eigen vectors of A. But this
does not seem to be strictly true and seems to require specific
eigenvectors, and I am not at all sure how these are computed.
(A %*% t(A) is required, BTW.) That is not the definition of the SVD. It is true that U are eigenvectors of A %*% t(A) and V of t(A) %*% A, but that does not make them left/right eigenvectors of A (unless that is your private definition). Since eigenvectors are not unique, it does mean that you cannot reverse the process, as you seem to be trying to do. Eigenvectors are only defined up to a sign (and more if there are duplicate eigenvalues) and singular vectors are only defined up to a sign (changing both U and V). You will find both vary by sign depending on the exact version of R used (including which BLAS and which compiler optimization level). Singular vectors have unit length, but eigenvectors do not have to (although they do in the code you have used).
Since W should have a zero row at the bottom, which when multiplied by U
will just remove the last column of U, I have just omitted the last row of
u from the outset:
eg, in R:
a <- matrix(c(c(1,2,3),c(5,14,11)),3,2)
u <- eigen(a %*% t(a))$vectors[,1:2]
v <- eigen(t(a) %*% a)$vectors
w <- sqrt(diag(eigen(t(a) %*% a)$values))
u %*% w %*% t(v)
gives:
[,1] [,2]
[1,] -0.9390078 -5.011812
[2,] -3.3713773 -13.734403
[3,] -1.3236615 -11.324660
which seems a little off the mark.
It is not expected to work.
The value for v is:
[,1] [,2]
[1,] 0.1901389 0.9817572
[2,] 0.9817572 -0.1901389
Where as svd(a)$v is:
[,1] [,2]
[1,] -0.1901389 0.9817572
[2,] -0.9817572 -0.1901389
If I substitute this in the above, I get:
u %*% w %*% t(svd(a)$v)
which returns:
[,1] [,2]
[1,] 1 5
[2,] 2 14
[3,] 3 11
which is what the SVD should do. I assume there is some rule about setting
the signs on eigenvectors for SVD, and would appreciate any help.
There is no rule: the SVD is computed by a different algorithm.
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595