2

I am trying to differentiate the following function, with respect to a matrix $X$:

$$ \operatorname{tr}(AX(X^TX)^{-1}B) $$

where tr corresponds to the trace. Is there an easy way to see what the derivative will be? I've come across rules for $\operatorname{tr}(AXB)$ but not the form above with inverses etc .

$A$ and $B$ are known (constant) matrices.

Michael Hardy
  • 1
NSR
  • 864
  • The function $Y \mapsto \operatorname{tr} (A Y B)$ is linear, so that part is straightforward. The function $X \mapsto X (X^T X)^{-1}$ is a pain, but not too bad if you factor and use the expansion $(I+Z)^{-1} = I-Z+Z^2...$). I mean expand $(X+H)((X+H)^T (X+H))^{-1}$ and find the terms that are linear in $H$. Or if you're feeling adventurous, use the chain and product rules. – copper.hat Jul 25 '14 at 06:50

1 Answers1

3

Define $W \equiv (X'X)^{-1}$, then $f = BAX : W$ and its derivative is $$ \frac {\partial f} {\partial X} = A'B'W - XW(BAX+X'A'B')W $$ The algebra to arive at this result is tedious but straight-forward. The only tricky part is knowing that $$ \eqalign { dW &= - W\,\,dY\,\,W \cr } $$ where $Y \!\equiv W^{-1}\!= X'X$

It's worth noting that both $Y$ and $W$ are symmetric.

Then you just expand the differential of $f$

$$ \eqalign{ df &= BA\,dX : W + BAX : dW \cr &= dX : (BA)'W - BAX : W\big[dX'X + X'dX\big]W \cr &= dX : A'B'W - WBAXW : \big[dX'X + X'dX\big] \cr &= A'B'W : dX - WBAXW : \big[dX'X + X'dX\big] \cr &= A'B'W : dX - WBAXW : dX'X - WBAXW : X'dX \cr &= A'B'W : dX - WBAXWX' : dX' - XWBAXW : dX \cr &= A'B'W : dX - XWX'A'B'W : dX - XWBAXW : dX \cr &=[A'B'W - XWX'A'B'W - XWBAXW]: dX \cr } $$ So the expression in brackets must be the derivative.

If you dislike the Frobenius product, you can carry out the above steps using the trace $$ \text{tr}(A'B) = A:B $$

lynne
  • 410