Prove identity matrix with singular value decomposition

Question

Let $I_d$ denote the $d \times d$ identity matrix and for $X \in \mathbb{R}^{m\times n}$ and $\lambda>0$ I have to prove the following:

$$(X^TX+\lambda I_n)^{-1}X^T=X^T(XX^T+\lambda I_m)^{-1}$$

I think about using Singular Value Decomposition of $X$, because of the $X^TX$ term, so it holds that:

$$X = U\Sigma V^T$$

But I dont know how to applied correctly.

Thanks in advance.

the SVD is not a bad idea. Why don't you try some additional steps in that direction? — Exodd, Jul 22 '20 at 16:22
I reach this result : $(V\Sigma^T \Sigma V^T + \lambda I_n)^{-1}X^T=X^T(U\Sigma \Sigma^T U^T +\lambda I_m)^{-1}$ but i get stuck. — Hojas, Jul 22 '20 at 16:28
good. Now notice that $(VAV^T)^{-1} = \overline VA^{-1}\overline V^T$ and continue — Exodd, Jul 22 '20 at 16:42
the complex conjugate of $V$, but since we are on the real field, it is equal to $V$ — Exodd, Jul 22 '20 at 16:57
@stochastic I put wrong the dimension of $X$, now it is correct, thanks! — Hojas, Jul 22 '20 at 17:36

score 0 · Accepted Answer · edited Jul 22 '20 at 17:47

$\newcommand{\diag}{\operatorname{diag}}$You are on the right track using SVD. Just need to carry some annoying algebra: $$ \begin{align} (X^TX+\lambda I_n)^{-1}X^T&=X^T(XX^T+\lambda I_m)^{-1}\\ \iff (V\Sigma^T U^T U\Sigma V^T + \lambda I_n)^{-1} V\Sigma^T U^T &= V\Sigma^T U^T (U\Sigma V^T V\Sigma^T U^T + \lambda I_m)^{-1}\\ \iff (V(\Sigma^T\Sigma + \lambda I_n)V^T)^{-1} V\Sigma^T U^T &= V\Sigma^T U^T (U(\Sigma\Sigma^T + \lambda I_m) U^T)^{-1}\\ \iff V(\Sigma^T\Sigma + \lambda I_n)^{-1}V^T V\Sigma^T U^T &= V\Sigma^T U^T U(\Sigma\Sigma^T + \lambda I_m)^{-1} U^T\\ \iff V(\Sigma^T\Sigma + \lambda I_n)^{-1}\Sigma^T U^T &= V\Sigma^T (\Sigma\Sigma^T + \lambda I_m)^{-1} U^T\\ \end{align} $$ Now all that is left is to show that $(\Sigma^T\Sigma + \lambda I_n)^{-1}\Sigma^T=\Sigma^T (\Sigma\Sigma^T + \lambda I_m)^{-1}$. Let us explicitly calculate these matrices:

Without loss of generality assume $m < n$, then $\Sigma^T\Sigma = \diag(\sigma_1^2,\dots,\sigma_m^2, 0,0,\dots)$ and $\Sigma\Sigma^T = \diag(\sigma_1^2,\dots,\sigma_m^2)$. Using the fact that inverse of a diagonal matrix is a diagonal matrix with it's diagonal elements inverted, we have $$(\Sigma^T\Sigma + \lambda I_n)^{-1}= \diag^{-1}(\sigma_1^2+\lambda,\dots,\sigma_m^2+\lambda, \lambda,\lambda,\dots) = \diag((\sigma_1^2+\lambda)^{-1},\dots,(\sigma_m^2+\lambda)^{-1}, \lambda^{-1},\lambda^{-1},\dots).$$ Multiplying on left by the $n\times m$ diagonal matrix $\Sigma^T$, we have $$(\Sigma^T\Sigma + \lambda I_n)^{-1}\Sigma^T= \diag_{n\times m}(\sigma_1(\sigma_1^2+\lambda)^{-1},\dots,\sigma_m(\sigma_m^2+\lambda)^{-1}).$$ Note that the extra $\lambda^{-1}$s got killed by the zero rows of $\Sigma^T$.

Similarly for the right hand side: $$(\Sigma\Sigma^T + \lambda I_m)^{-1} = \diag^{-1}(\sigma_1^2+\lambda,\dots,\sigma_m^2+\lambda) = \diag((\sigma_1^2+\lambda)^{-1},\dots,(\sigma_m^2+\lambda)^{-1}).$$ Multiplying on the left by the $n \times m$ diagonal matrix $\Sigma^T$ we have: $$\Sigma^T(\Sigma\Sigma^T + \lambda I_m)^{-1} = \diag_{n\times m}(\sigma_1(\sigma_1^2+\lambda)^{-1},\dots,\sigma_m(\sigma_m^2+\lambda)^{-1}).$$ This concludes the proof.

Prove identity matrix with singular value decomposition

1 Answers1