Proof of Eckart-Young-Mirsky theorem -- Revisited

Question

This is a question about an answer in the forum. Th question was originally about Eckart-Young-Mirsky theorem proof. The first answer, still, very concise and I have some questions about. There were some discussions in the comment but I still cannot get answers for my questions. Here is the answer:

Since $\mathrm{rank}(B)=k$, $\dim\mathcal{N}(B)=n-k$ and from $$\dim\mathcal{N}(B)+\dim\mathcal{R}(V_{k+1})=n-k+k+1=n+1$$ (where $V_{k+1}=[v_1,\ldots,v_{k+1}]$ is the matrix of right singular vectors associated with the first $k+1$ singular values in the descending order), we have that there exists an $$x\in\mathcal{N}(B)\cap\mathcal{R}(V_{k+1}), \quad \|x\|_2=1.$$ Hence $$ > \|A-B\|_2^2\geq\|(A-B)x\|_2^2=\|Ax\|_2^2=\sum_{i=1}^{k+1}\sigma_i^2|v_i^*x|^2\geq\sigma_{k+1}^2\sum_{i=1}^{k+1}|v_i^*x|^2=\sigma_{k+1}^2. > $$ From $\|A-A_k\|_2=\sigma_{k+1}$, one gets hence $\|A-B\|_2\geq\|A-A_k\|_2$. No contradiction required.

1. I do not understand why $x\in\mathcal{N}(B)\cap\mathcal{R}(V_{k+1}) \ne {0}?$
2. I do not understand $\|Ax\|_2^2=\sum_{i=1}^{k+1}\sigma_i^2|v_i^*x|^2.$ Is it because that $v_i^Tx = 0, i> k +1$. If yes, where does the absolute value come from?
3. I do not understand $\sigma_{k+1}^2\sum_{i=1}^{k+1}|v_i^*x|^2=\sigma_{k+1}^2.$ I think this is related to the assumption that the norm of $x$ is 1, but not sure how to rigorously show this.

---

Answer 1

1. Choose bases in both $\mathcal{N}(B)$ and $\mathcal{R}(V_{k+1})$. Take the vectors from both bases, that's going to be $n+1$ vectors. So that system cannot be linearly independent. Hopefully, you can show that there're non-zero vectors in $\mathcal{N}(B) \cap \mathcal{R}(V_{k+1})$ from that.

2. $Ax = U \Sigma V^* x$. But multiplying a vector by $V^* = V^{-1}$ is the same as representing it in the basis formed by the columns of $V$. $x \in \mathcal{R}(V_{k+1})$ and therefore its coordinates after the $k+1$ would be $0$. So the coordinates of the vector $V^* x$ are $v_1^* x, v_2^* x, \ldots, v_{k+1}^* x, 0, \ldots, 0$. After multiplication by $\Sigma$ the coordinates would become $\sigma_1 v_1^* x, \sigma_2 v_2^* x, \ldots, \sigma_{k+1} v_{k+1}^* x, 0, \ldots, 0$. Then we multiply be $U$, but it's unitary and doesn't change the norm.

3. We've already shown that the coordinates of $V^* x$ are $v_1^* x, v_2^* x, \ldots, v_{k+1}^* x, 0, \ldots, 0$. So the norm of that vector is $\sum_{i=1}^{k+1} |v_i^* x|^2$. But $V$ is unitary, and so is $V^*$, so after applying it to the vector of norm 1, you would get a vector of norm 1 again.