I was wondering how one computes the derivative of the inverse of a matrix wrt to the original matrix. In my case, it might even be simpler as the matrix is an invertible diagonal matrix.
So, $X$ is a diagonal matrix and I would like to compute the derivative $\frac{dX^{-1}}{dX}$
I was wondering if the result should be the diagonal matrix, but could not convince myself why that should be.