1

Let $A\in \mathbb R^{5\times 3}$ with left singular vectors $u_1,\ldots,u_5$, right singular vectors $v_1,\ldots,v_3$ and singular values $\sigma_1,\sigma_2,\sigma_3=4,2,1$

  • Suppose $y=2u_1+3u_2+4u_3+5u_5$. Find the minimum $ \min\limits_x\Vert y-Ax\Vert_2^2$

  • Find $\hat x=\operatorname*{arg min}\limits_x \|y-Ax\|_2^2$. Write $\hat x$ as a linear combination of the vectors $v_j$

  • Is the vector in part (b) the unique minimum? Explain

I know that $A$ can be written as $A=\left(\begin{matrix}|&|&|&|&|\\u_1&u_2&u_3&u_4&u_5\\|&|&|&|&|\end{matrix}\right)\left(\begin{matrix}4&0&0\\0&2&0\\0&0&1\\0&0&0\\0&0&0\end{matrix}\right)\left(\begin{matrix}-v_1^*-\\-v_2^*-\\-v_3^*-\end{matrix}\right)$ but that's about it. I was lamenting that A cannot be written in the form $\sum_i\sigma_iu_iv_i^*$. I wrote $y-Ax=2u_1+3u_2+4u_3+5u_5-U\Sigma V^*x$, then I tried to compute $U\Sigma V^*x=4v_1^*xu_1+2v_2^*xu_2+v_3^*xu_3$, and then equating the terms, to get $\begin{cases}2=4v_1^*x\\3=2v_2^*x\\4=v_3^*x\end{cases}$, but not sure this helps. Anyone have any ideas?

Michael Hardy
  • 1
Vons
  • 11,004
  • If $A^TA$ is non$-$singular then $\hat{x}$ is unique and equals $\hat{x}=(A^TA)^{-1}A^Ty$. Write $A$ as $U\Sigma V^T$ and see how $(A^TA)^{-1}A^T=V(\Sigma^T \Sigma)^{-1}\Sigma^TU^T$. You can also express $y$ in the form $Uz$ where $z=(2,3,4,0,5)^T$ so $\hat{x}$ simplifies to $$\hat{x}=V(\Sigma^T \Sigma)^{-1}\Sigma^Tz$$ – Matthew H. Jan 31 '21 at 04:42
  • Oh! Ordinary least squares in disguise? Hahahaha thank you – Vons Jan 31 '21 at 04:56

0 Answers0