2

Ok so I remember this trick that you could sometimes use to solve $Mx = y$ for $x$, given $M$ and $y$. It did not invovle computing $M^{-1}$. Can you remind me what it was?

The only thing I remember is that you would do something to the equation $Mx = y$ such that you would get a new equation $$M' x = y'$$ where $M'$ and $y'$ are different from before. And then, it turns out that $M'$ is always invertible, and then you could find the inverse.

What was that trick?

EDIT: I know there are million times more efficient ways to do this that does not involve finding inverses. Not asking about that.

Imean H
  • 211
  • 1
  • 4
  • You can't “solve” a singular system, as you point out. But the least-squares method provides the closest approximation to a solution. – Matthew Leingang Jun 14 '17 at 15:53

1 Answers1

4

My best guess is that you're thinking of linear least-squares. Given the (usually overdetermined) system $Mx = y$, we can "multiply both sides by $M^T$" to get the least-squares equation $$ (M^TM)\hat x = (M^Ty) $$ and a solution $\hat x$ is known as a least-squares solution to $Mx = y$. If the columns of $M$ are linearly independent (i.e. if $M$ has full column-rank), then $M^TM$ is invertible and we have the unique least-squares solution $$ \hat x = (M^TM)^{-1}(M^Ty) $$ It is not necessarily true that $M' = M^TM$ is invertible. However, it is always true that this new system of equations will necessarily have at least one solution.

Notably, the Moore-Penrose pseudoinverse calculates "inverses" to singular matrices with this least-squares idea in mind. In particular, if $Mx = y$, then $M^+y$ is the least-squares solution $\hat x$ such that $\|\hat x\|$ is minimized.

Ben Grossmann
  • 225,327