According to Wolfram Alpha, $\begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}^{-1}\begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}\neq \begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}^{-1}$.
Links to the two computations:
MatrixPower[{{3, 1}, {5, 2}}, -1] {{1, 0}, {0, 1}}
MatrixPower[{{3, 1}, {5, 2}}, -1]
To further confuse things, leaving out the inverse produces the expected result ($\begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}\begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}= \begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}$):