1

Assuming that the following are matrices with their corresponding dimensions: $\mathbf{X}\in\mathbb{R}^{p\times d},\mathbf{A}\in \mathbb{R}^{p\times p}$ which is symmetric and $\mathbf{B} \in \mathbb{R}^{(p-d)\times (p-d)}$, then what is the derivative: $\frac{\partial}{\partial \mathbf{X}}\log\big|\mathbf{X}^T\mathbf{A}\mathbf{X}\mathbf{B}\big|$ Derivation: $\frac{\partial }{\partial X}\log\big|\mathbf{X}^T\mathbf{A}\mathbf{X}\mathbf{B}\big|=\frac{1}{\big|\mathbf{X}^T\mathbf{A}\mathbf{X}\mathbf{B}\big|}\frac{\partial}{\partial\mathbf{X}}\big|\mathbf{X}^T\mathbf{A}\mathbf{X}\mathbf{B}\big| $

J.C
  • 11
  • What do you denote by $|A|$? The determinant $\det A$? Or the absolute part of $A$, i. e. $(A^t A)^{1/2}$? – martini Nov 11 '16 at 08:38
  • 1
    @martini Looks like $|A|$ is $\operatorname{det} A$. Otherwise $\frac1{|\ldots|}$ would make no sense. – Tobias Nov 11 '16 at 09:00
  • @Tobias Thx, of course. – martini Nov 11 '16 at 09:01
  • $XB$ and the determinant $|X^T A X B|$ only make sense if $B\in \mathbb{R}^{d\times d}$. – Tobias Nov 11 '16 at 09:02
  • @J.C. Use $\partial \det(A)[H] = \det(A)\mathrm{tr}(A^{-1}H)$ and the chain rule. – martini Nov 11 '16 at 09:04
  • See also the discussions on the interpretations of $\partial/(\partial X)$ there: http://math.stackexchange.com/questions/2007942/derivative-of-a-matrix-trace-w-r-t-complex-matrix. – Tobias Nov 11 '16 at 09:07
  • You are right about $\mathbf{B} \in \mathbb{R}^{(p-d)\times(p-d)}$ I have a mistake, the derivative is the following: $\frac{\partial}{\partial\mathbf{X}}\log\big|\mathbf{X}^T\mathbf{A}\mathbf{X}\big|\big|\mathbf{V}^T\mathbf{C}\mathbf{V}\big|$ were the second derivative is independent of X and therefore there is a result for this, thank you all for your help. – J.C Nov 11 '16 at 09:17
  • That means you have essentially $\delta \log|X^T A X| = \frac{\operatorname{det}(X^T A X)\operatorname{tr}((X^T A X)^{-1}\cdot(\delta X A X + X A \delta X))}{|X^T A X|}$. The derivative of the additive part $\log|V^T C V|$ is zero. – Tobias Nov 11 '16 at 11:07

1 Answers1

2

Let $M = X^TAX\,\,$ and write the function in terms of this new variable, then find its differential and gradient $$\eqalign{ f &= \log \det M \cr\cr df &= d(\log\det M) \cr &= d({\rm tr}\log M) \cr &= M^{-T}:dM\cr &= M^{-T}:d(X^TAX)\cr &= M^{-T}:(dX^TAX + X^TA\,dX)\cr &= M^{-1}:(AX)^TdX + M^{-T}:X^TA\,dX \cr &= AXM^{-1}:dX + A^TXM^{-T}:dX \cr &= (AXM^{-1} + A^TXM^{-T}):dX \cr\cr \frac{\partial f}{\partial X} &= AXM^{-1} + A^TXM^{-T} \cr }$$where colons denote the Frobenius Inner Product.

Since $A$ is symmetric, $M$ is too, and the gradient can be simplified to $$\eqalign{ \frac{\partial f}{\partial X} &= 2AXM^{-1} \cr &= 2AX(X^TAX)^{-1} \cr }$$

greg
  • 35,825