2

I have read that the Moore-Penrose inverse $A^+$ of a matrix $A$ is the same as the standard inverse $A^{-1}$ in the case $A$ is a square, invertible matrix. Is there any relation between the Moore-Penrose inverse and the standard inverse in the case $A$ is a non-invertible square matrix?

The reason I am asking is due to implementing an inverse for a large matrix in R. When I tried the standard inverse, I found out the matrix was non-invertible. But when I tried to find the Moore-Penrose inverse, I got a new matrix.

Comp_Warrior
  • 262
  • @user1551 Standard inverse. I'll edit the question. – Comp_Warrior Jul 03 '13 at 11:05
  • 7
    That doesn't make sense. If $A$ is noninvertible, it has no standard inverse, so you cannot speak of the relation between $A^+$ and a non-existing entity. – user1551 Jul 03 '13 at 11:07
  • Any ideas why a non-invertible matrix could have a pseudoinverse? (as calculated by R - I used the ginv function in the MASS package) – Comp_Warrior Jul 03 '13 at 11:10
  • The standard inverse is a particular case of the Moore-Penrose Inverse. All matrices have its Moore-Penrose Inverse, and when the matrix is square and non-singular, then it is equivalent to the standard inverse. Just check http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse for further properties. –  Jul 03 '13 at 11:10
  • @noether I need to put the inverse of a matrix in an expression. If its standard inverse does not exist, do you think using the Moore-Penrose inverse would be ok? – Comp_Warrior Jul 03 '13 at 11:30
  • @Comp_Warrior Look at the answer of user1551 below. I guess you'd better expose your actual problem, because the pseudo inverse and the standard inverse are different things. For a particular case both matrices are the same, but the meaning of both matrices is different. –  Jul 03 '13 at 11:35

2 Answers2

8

The Moore-Penrose pseudoinverse $A^+$ has many brilliant properties. First of all, it coincides with the standard inverse $A^{-1}$ when $A^{-1}$ exists. I found, in particular, interesting the use of $A^+$ in relation with systems of linear equations $A\mathbf x=\mathbf b$:

  1. If $A$ is $n\times n$ and invertible, the unique solution is $\mathbf x=A^{-1}\mathbf b=A^{+}\mathbf b$. The following case is more interesting.
  2. If $A$ is non invertible (because $|A|=0$ or $A$ is rectangular and, hence $A^{-1}$ does not exists) then the system can have no or infinite solutions.

No solution: the vector $A^{+}\mathbf b$ is the closest approximation to a solution in least squares sense. In other words, the sum of squared deviations from $b$ is minimal or $||A\mathbf x-b||_2$ is minimal when $\mathbf x=A^{+}\mathbf b$.

Infinite number of solutions: the vector $A^{+}\mathbf b$ is, among all solutions, the one with the shortest euclidean norm (i.e., it is as close to the origin as it is possible)

To sum up, using the pseudoinverse gives the same solution when this is unique. When there is no solution, it still can be used to obtain a reasonable approximation of the solution. Finally, when many solutions are available (underdetermined system), the one produced is the "smallest". Hope this helps.

user1783444
  • 546
  • 1
    This makes sense. But I'm looking at a problem where I have to evaluate something like : (inverse of matrix)*(vector). (This is not for a system of equations.) The matrix turns out to be non-invertible. What interpretation would you give to this expression if the inverse was replaced by the Moore-Penrose inverse? – Comp_Warrior Jul 03 '13 at 13:15
  • The interpretation I can give to you is: take care with operations you are doing along a math problem. Seriously, interpreting the problem again with caution is the best thing you could do whenever this type of non-existing operations are being done. I consistently do this type of thing as well to not to stop my reasoning along a good attempt, but after all done, steps backwards is needed. – R. W. Prado Aug 17 '21 at 07:01
5

Moore-Penrose pseudoinverse can be obtained from singular value decomposition. For every $m\times n$ tall matrix ($m>n$) real or complex $A$, there is always a singular value decomposition $$ A=U_{m\times m}\ \Sigma_{m\times n}\ V^\ast_{n\times n} =U\pmatrix{\sigma_1\\ &\sigma_2\\ &&\ddots\\ &&&\sigma_k\\ &&&&0\\ &&&&&\ddots\\ &&&&&&0\\ 0&\cdots&\cdots&\cdots&\cdots&\cdots&0\\ \vdots&&&&&&\vdots\\ 0&\cdots&\cdots&\cdots&\cdots&\cdots&0\\ }V^\ast $$ where $U,V$ are unitary matrices (which can be taken as real orthogonal matrices when $A$ is real), $k=\mathrm{rank}(A)$ and $\sigma_1\ge\ldots\ge\sigma_k>0$. Then $A^+$ is defined as $$ A^+=V\Sigma^+U^\ast=V\pmatrix{ \sigma_1^{-1}&&&&&&&0&\cdots&0\\ &\sigma_2^{-1}&&&&&&\vdots&&\vdots\\ &&\ddots&&&&&\vdots&&\vdots\\ &&&\sigma_k^{-1}&&&&\vdots&&\vdots\\ &&&&0&&&\vdots&&\vdots\\ &&&&&\ddots&&\vdots&&\vdots\\ &&&&&&0&0&\cdots&0\\ }U^\ast. $$ When $A$ is wide (i.e. $m<n$), the singular value decomposition and $A^+$ have analogous forms (just transpose everything). If $A$ is square, both $\Sigma$ and $\Sigma^+$ are square diagonal matrices (so that the two large zero blocks in the above are void). Furthermore, when $A$ is invertible, we have $k=\mathrm{rank}(A)=n$ and hence $\Sigma=\mathrm{diag}(\sigma_1,\ldots,\sigma_n)$ and $\Sigma^+=\mathrm{diag}(\sigma_1^{-1},\ldots,\sigma_n^{-1})=\Sigma^{-1}$. It follows that $A^+=V\Sigma^+U^\ast=V\Sigma^{-1}U^\ast=(U\Sigma V^\ast)^{-1}=A^{-1}$.

user1551
  • 139,064