How do I differentiate a Kronecker product with respect to a scalar (a matrix element)?

Question

I am developing in my PhD an new proposed approach using SEM (stochastic expectation maximization algorithm) that fits a multivariate linear mixed model by maximizing the likelihood function to find MLE. The form of the LF is highly complicated due to the large dimension of parameters which requires using numerical techniques based on deriving the Hessian matrix that reach up to size 36X36. Finding this matrix require finding the first and second derivatives of many complicated function such as Kronecker product, determinate, inverse of determinate, inverse of a matrix and so on. My question here related to deriving one of those quantities.

How I differentiate $\large\frac{\partial{(\Sigma \otimes I_n)}}{\partial{\sigma_j}}$

where

$\Sigma$ is a symmetric positive definite matrix of order $n$
It is known that $\large\frac{\partial{\Sigma^{-1}}}{\partial{\sigma_j}} = F_j$ , where $\Sigma^{-1}= \sum_{i=1}^{h} \sigma_j F_j$ and $\sigma_j$'s is the non-redundant elements of the precision matrix $\Sigma^{-1}$
$I_n$ is the identity matrix of order $n$

Welcome to MSE. Can you please tell us what have you tried? We can help you easily. — , Dec 21 '17 at 20:46
I re-edit the question clarifying precisely to what problem it was related. The first post was not clear or it seemed to be off-topic because it was not detailed enough as I was a new member in MSE. The answer was posted here was so valuable that I applied in my computer code which I finish. In fact MSE community was of my priority to first search under its vast information of questions and answers. I expect it should be opened because it might be beneficial to anyone like it was to me. — N. I. ElZayat, Nov 24 '18 at 19:57

score 0 · Accepted Answer · answered Dec 22 '17 at 20:16

0

For an invertible matrix $\Sigma$ we have $$\eqalign{ I &= \Sigma\Sigma^{-1} \cr 0 &= d\Sigma\,\Sigma^{-1} + \Sigma\,d\Sigma^{-1} \cr d\Sigma &= -\Sigma\,\,d\Sigma^{-1}\,\Sigma \cr }$$ Also $$d(\Sigma\otimes I) = (d\Sigma\otimes I)$$ Now substitute the specifics of your matrix into these expressions.

answered Dec 22 '17 at 20:16

lynn

3,396

This implies that the right side of the last equation will be $ \ (-\Sigma d\Sigma^{-1} \Sigma) \otimes I = (-\Sigma F_l \Sigma) \otimes I$ – N. I. ElZayat Dec 23 '17 at 04:06
can you help on simplifying the last form – N. I. ElZayat Dec 23 '17 at 04:15
It could be $(\Sigma \otimes I) F_j (\Sigma \otimes I)$ – N. I. ElZayat Dec 23 '17 at 04:22
You can choose either $$\eqalign{&(-\Sigma F_j\Sigma)\otimes I\cr&(-\Sigma\otimes I)(F_j\otimes I)(\Sigma\otimes I)\cr}$$ as the simplified expression. – greg Dec 23 '17 at 18:13

How do I differentiate a Kronecker product with respect to a scalar (a matrix element)?

1 Answers1