0

How to calculate $\frac{\partial \mathrm{logdet}(I+XX^T)}{\partial X}$? ($X\in\mathbb{R}^{n\times m}$)

I know that $\frac{\partial \mathrm{logdet}(I+X)}{\partial X}=(I+X)^{-T}$, can we use it?

Lee
  • 1,910
  • 12
  • 19
  • 2
    Try the chain rule, starting from your second equation with $\log \det (I + U)$ where $U = X X^T$. – John Barber Nov 17 '19 at 13:44
  • @JohnBarber $\frac{\partial \mathrm{logdet}(I+XX^T)}{\partial XX^T}\frac{\partial XX^T}{\partial X}$, first derivative is known, but how to differentiate second derivative, which is differentiate matrix by matrix, =2X? – Lee Nov 17 '19 at 13:59
  • When in doubt in these types of matrix problems, express everything with indices: $\partial (X X^T) / \partial X$ becomes $\partial \left(\sum_m X_{km}X_{\ell m}\right) / \partial X_{ij}$. Do you know what $\partial X_{km} / \partial X_{ij}$ is? – John Barber Nov 17 '19 at 14:03

1 Answers1

1

For typing convenience, define the symmetric matrix $$\eqalign{A &= I + XX^T}$$ Write the cost function in terms of this matrix.
Then calculate its differential and gradient. $$\eqalign{ \phi &= \log\det A \\ d\phi &= A^{-T}:dA \\ &= A^{-T}:(dX\,X^T + X\,dX^T) \\ &= (A^{-T}+A^{-1}):dX\,X^T \\ &= 2A^{-1}X:dX \\ \frac{\partial \phi}{\partial X} &= 2A^{-1}X \\ }$$ where a colon has been used to denote the trace/Frobenius product, i.e. $$\eqalign{A:B = {\rm Tr}(A^TB)}$$ and the cyclic property allows terms in a trace product to be rearranged $$\eqalign{ A:BC &= BC:A \\&= B:AC^T \\&= A^TB:C^T \\&= etc\\ }$$

greg
  • 35,825