How to prove these 2 matrix problems?

Question

1. I'm reading a book and it gives that $\frac{\partial}{\partial A}Tr(AB)=B^T$, then it shows we can obtain $\frac{\partial}{\partial A}Tr(ABA^T)=A(B+B^T)$. But it seems we should have $\frac{\partial}{\partial A}Tr(ABA^T)=AB^T$ accordingly. Am I wrong or the book?

2. The book also gives another expression: $\frac{\partial}{\partial x}ln|A|=Tr(A^{-1})\frac{\partial A}{\partial x}$ , how to prove that?

Answer 1

So we look at the map $\def\Tr{\mathop{\rm Tr}}f \colon A \mapsto \Tr(AB)$. We have \begin{align*} f(A+H) &= \Tr(AB + HB)\\ &= \Tr(AB) + \Tr(HB)\\ &= f(A) + \Tr(HB) \end{align*} So $$ f'(A)H = \Tr(HB) $$ (as obviously for linear maps). If we want to calculate the derivatives with respect to the entries of $A$, let $E_{ij}$ denote the $(i,j)$-th unit matrix (the only nonzero entry is $(i,j)$, which is 1). Then $$ \partial_{a_{ij}}f(A) = f'(A)E_{ij} = \Tr(E_{ij}B) =\sum_{k,l=1}^n [E_{ji}]_{k,l}[B]_{l,k} = [B]_{j,i} $$ So if we denote the matrix of these partial derivatives by $\partial_A f$, we have $\partial_A f = B^t$.

Now consider $g(A) = \Tr(ABA^t)$. Note that $g$ is no longer, linear, but quadratic in $A$. Along the same lineas as above, \begin{align*} g(A+H) &= \Tr(ABA^t) + \Tr(HBA^t) + \Tr(ABH^t) + \Tr(HBH^t)\\ &= g(A) + \Tr(AB^tH^t) + \Tr(ABH^t) + o(\|H\|)\\ \end{align*} So $$ g'(A)H = \Tr\bigl(A(B^t+B)H\bigr) $$ As above, we have $$ \partial_{a_{ij}}g(A) = g'(A)E_{ij} = \Tr\bigl(A(B^t+B)E_{ji}\bigr) = [A(B^t+B)]_{ij} $$ So $\partial_A g(A) = A(B^t+B)$, so the book is right.

Regarding your last question, what is $x$? What can be said, is: $$ \det A = \sum_{i=1}^n (-1)^{i+j} a_{ij}\det A_{ij} $$ holds for each $j$, where $A_{ij}$ results from $A$ removing the $i$-row and $j$-th column. Hence with $h(A) := \log\det A$ \begin{align*} \partial_{a_{ij}}h(A) &= \frac{1}{\det A} \det A_{ij} \end{align*} Now recall, that $\det A \cdot (A^{-1})_{ij} = \det A_{ji}$ (by Cramer's formula), hence $$ \partial_{a_{ij}}h(A) = \frac{1}{\det A} \det A_{ij} = (A^{-1})_{ji} $$ Hence we have $\partial_A h(A) = (A^{-1})^t$.

Answer 2

1) You can do a quick sanity check using $1\times 1$ matrices, i.e. scalars. The equation reduces to: $$\frac {d(a^2 b)}{da} = 2 a b$$ The result in the book is compatible with the scalar version, yours isn't.

2) Can't explain it any better than this Wikipedia entry.