Differentiability involving Det function

Question

I've been struggling on this for a while, and so any help will be great. I've just dont seem to handle matrix manipulation well and im not sure if my first part to the question is correct (rigour wise), onwards I'm not really advancing.

Answer 1

The derivative is smooth since it is the sum and product of monomials. Hence we can just compute the derivative component by component.

The derivative is multilinear as a function of its columns. I will use the notation $H= \begin{bmatrix} h_1 & \cdots & h_n \end{bmatrix}$, and $d(x_1,...,x_n) = \det \begin{bmatrix} x_1 & \cdots & x_n \end{bmatrix} = \det X$.

Then $\det I = d(e_1,...,e_n) = 1$, $d(e_1+\lambda h_1,...,e_n) = 1+\lambda d(h_1, e_2,...,e_n) = 1+\lambda [H]_{11}$.

Hence $D \det(I)(h_1 e_1^T) =[H]_{11} $.

A similar analysis shows that $D \det(I)(h_k e_k^T) =[H]_{kk} $.

Since $H = \sum_k h_k e_k^T$, we have $D \det(I)(H) = D \det(I)(\sum_k h_k e_k^T) = \sum_k D \det(I)(h_k e_k^T) = \sum_k [H]_{kk} = \operatorname{tr} H$.

Now suppose $A$ is invertible. Note that $\det (A+H) = \det A \det (I+ A^{-1} H)$, from which it follows that $D \det(A)(H) = (\det A) D \det(I) (A^{-1} H)$. This gives $D \det(A)(H) = (\det A) \operatorname{tr} ( A^{-1} H ) $.

Computing $D^2 \det$ is straightforward, but a bit of a notational nightmare. We know that $A \mapsto \det A$ is smooth, so this gives some options.

One way is to let $g(A) = (H \mapsto D \det(A)(H))$, that is, $g(A)$ is the function that maps $H$ to $D \det(A)(H))$, and find $Dg(A)$. This is straightforward, but many people find the function baggage ($H \mapsto ...$) a bit disconcerting or confusing.

Another way is to fix $H$, and let $g_H(A) = D \det(A)(H)$, and compute $Dg_H(A)$.

We have $g_H(A) = (\det A) \operatorname{tr} (A^{-1} H)$. If we let $g_1(A) = \det A$ and $g_2(A) = \operatorname{tr} (A^{-1} H)$, then we have $g_H(A) = g_1(A) g_2(A)$, and so $Dg_H(A)(\Delta) = g_1(A) D g_2(A)(\Delta) + D g_1(A) (\Delta) g_2(A)$ by the product rule.

We have already computed $Dg_1(A) (\Delta) = (\det A) \operatorname{tr} (A^{-1} \Delta)$.

To compute $D g_2(A)$, we note that $\operatorname{tr}$ is linear, so we need to see how $A \mapsto A^{-1} H$ behaves when perturbed. For suitably small $\Delta$, we have $(A+\Delta)^{-1} = ((I+\Delta A^{-1}) A )^{-1} = A^{-1}(I -\Delta A^{-1} + (\Delta A^{-1})^2- \cdots)$, from which we can read off $Dg_2(A)(\Delta) = -\operatorname{tr} (A^{-1} \Delta A^{-1}H)$.

Combining gives $Dg_H(A)(\Delta) = (\det A)\left(\operatorname{tr} (A^{-1} \Delta) \operatorname{tr} (A^{-1} H) - \operatorname{tr} (A^{-1} \Delta A^{-1}H) \right)$.

Consequently, we have $D^2 \det(A)(\Delta)(H) = (\det A)\left(\operatorname{tr} (A^{-1} \Delta) \operatorname{tr} (A^{-1} H) - \operatorname{tr} (A^{-1} \Delta A^{-1}H) \right)$.