I have trouble in calculating partial derivatives of matrix recently. The problem is described as following \begin{align} \frac{\partial \log \det\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)}{\partial \mathbf{Q}} \end{align} where $\mathbf{I}$ is identity matrix. $\mathbf{\Sigma}$ and $\mathbf{Q}$ are square matrix. $\det(\cdot)$ is determinant operation. I wanna compute this formula. thanks a lot.
Asked
Active
Viewed 175 times
-3
-
Hint: use the fact that the derivative of $\log \det X$ with respect to $X$ is $X^{-1}$ and apply the chain rule. – Brian Borchers Sep 04 '18 at 02:29
-
@Qiuyun Kindly show some of the things you may have tried or your thoughts about the problem. As it stands, the question would be considered lacking in effort by the community and is likely to be downvoted and closed. – user0 Sep 04 '18 at 04:17
-
@BrianBorchers Thank you for your answer! I have solved this formula and the result is given by next answer. – Qiuyun.Zou Sep 04 '18 at 09:34
-
@ DevashishKaushik Thank you for your answer. I will keep it in mind and show more information I have thought. – Qiuyun.Zou Sep 04 '18 at 09:39
2 Answers
1
Define the matrix $$X=(I+\Sigma Q)$$ Write the function in terms of this new matrix, then find the differential and gradient. $$\eqalign{ \lambda &= \log\det X \cr d\lambda &= X^{-T}:dX = X^{-T}:\Sigma\,dQ = \Sigma^TX^{-T}:dQ \cr \frac{\partial\lambda}{\partial Q} &= \Sigma^TX^{-T} \cr }$$ Since $(\Sigma, Q)$ are symmetric, the gradient can be simplified to $$\frac{\partial\lambda}{\partial Q} = \Sigma\,(I+Q\Sigma)^{-1}$$
greg
- 35,825
1
Solution: \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}= \left[\begin{array}{ccc} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{11}}&\cdots &\frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{1,u}}\\ \vdots & \ddots &\vdots\\ \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{u,1}} &\cdots & \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{u,u}} \end{array} \right] \end{align} So the key is to find $\frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}}$ for $(i,j)\in [u,u]$.
To this end,we apply the fact \begin{align} \frac{\partial \log \det (\mathbf{A})}{\partial x}=\text{tr}\left({\mathbf{A}^{-1}\frac{\partial \mathbf{A}}{\partial x}}\right) \end{align} where $\mathbf{A}$ is function of $x$, i.e., $\mathbf{A}=\mathbf{A}(x)$. (The proof of above formula can be given by $\frac{\partial \log \det(A)}{\partial x}=\frac{1}{\det(\mathbf{A})}\frac{\partial \det(\mathbf{A})}{\partial x}$ and Jacobi's formula).
With this knowledge, we have \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}} &\overset{(a)}{=}\text{tr} \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}\boldsymbol{e}_i\boldsymbol{e}_j^T}\right)\\ &\overset{(b)}{=}\boldsymbol{e}_j^T \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}}\right) \boldsymbol{e}_i \end{align} where $\boldsymbol{e}_i$ is a vector with all zero elements expect i-th element being 1 in step $(a)$, and $(b)$ holds via the property of trace operation.
Here, $\mathbf{\Sigma}$ and $\mathbf{Q}$ are symmetric matrix. We then have \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial Q_{i,j}} =\boldsymbol{e}_i^T \left({\left({\mathbf{I}+\mathbf{\Sigma Q}}\right)^{-1}\mathbf{\Sigma}}\right) \boldsymbol{e}_j \end{align} Thus, we can get \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}=(\mathbf{I}+\mathbf{\Sigma Q})^{-1}\mathbf{\Sigma} \end{align}
By the way, if $\mathbf{\Sigma}$ and $\mathbf{Q}$ are not symmetric, then the result is given by \begin{align} \frac{\partial \log \det(\mathbf{I}+\mathbf{\Sigma Q})}{\partial \mathbf{Q}}=\mathbf{\Sigma}^T((\mathbf{I}+\mathbf{\Sigma Q})^{-1})^T \end{align}
Qiuyun.Zou
- 371