1

I thought I understood how to compute the singular value decomposition $A = U D V^T$ of a matrix $A$, based on this tutorial:

http://www.math.ucla.edu/~pskoufra/M33A-SingularValueDecompositionExample.pdf

To summarize, given a matrix $A$, you find the eigenvalues and eigenvectors of $A^T A$ and $A A^T$. The square roots of the eigenvalues are the singular values, and the eigenvectors make up the columns of $U$ and $V$.

However, when I tried to compute the SVD of the matrix $$ A = \left( \begin{matrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{matrix} \right), $$ I ran into some trouble. The singular values are $(1,1,0)$, and applying my method I got $$ U = \left( \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{matrix} \right), $$ $$ D = \left( \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \\ \end{matrix} \right), $$ $$ V = \left( \begin{matrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \\ \end{matrix} \right) $$ Indeed, everything seems to be fine. We have $A^T A = V D^T D V^T$, $A A^T = U D D^T U^T$, the eigenvalues and eigenvalues are all right, but there is a serious problem: $A \ne U D V^T$!

What have I done wrong?

Zur Luria
  • 1,514

1 Answers1

1

This is incorrect: and the eigenvectors make up the columns of U and V.

To get $V$ you take the eigenvectors of $A^TA\color{grey}{=\begin{pmatrix}0 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\end{pmatrix}}$, yes. And you get the correct $V$.

But $U$'s $i^{\text{th}}$ column is $\dfrac 1{\sigma _i}Av_i$, where $v_i$is $V$'s $i^{\text{th}}$ column, for all $i\in \{1,2\}$ (because of the rank of $A$).

So $U=\begin{pmatrix}0 & 1 & *\\ 1 & 0 & *\\ 0 & 0 & *\end{pmatrix}$. To find the remaining columns (in this case it's just one), complete the matrix in such a way that $U$ is orthogonal. One obvious choice is $U=\begin{pmatrix}0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1\end{pmatrix}$ and this works.

Git Gud
  • 31,356