0

I am having trouble finding the Jacobian and Hessian of this function involving the Kronecker product. I have a matrix $\mathbf{A}\in\mathbb{R}^{n\times n}$ and a vector $\mathbf{x}\in\mathbb{R}^{n^4}$. I have the function $f(\mathbf{A},\mathbf{x})=(\mathbf{A}\otimes\mathbf{A}\otimes\mathbf{A}\otimes\mathbf{A}) \mathbf{x}$. How do I find the Jacobian and Hessian of this function with respect to $\mathbf{A}$ and $\mathbf{x}$? I also have that $\mathbf{A}$ is orthogonal such that $\mathbf{AA}^T=I$, does this make the calculations easier?

Thanks for your help in advance

shex95
  • 73
  • 9
  • Hint: derivatives of Kronecker-products follow essentially the same product-rule as derivatives of normal product. I.e. $(A \otimes B)' = A'\otimes B + A\otimes B'$ – Simon Apr 30 '21 at 12:00
  • Hi Simon, thanks for your answer. Would I be right in saying $(A\otimes A)'=I\otimes A + A \otimes I$, where we are differentiating w.r.t to $A$ and $I$ is the identity? – shex95 Apr 30 '21 at 13:35
  • Yes, thats correct. Of course make sure you know what "differentiating w.r.t. a matrix" really means. Writing these things without explicit indices can be confusing. – Simon Apr 30 '21 at 13:44

1 Answers1

1

$\def\v{{\rm vec}}\def\E{{\cal E}}\def\F{{\cal F}}\def\G{{\cal G}}\def\H{{\cal H}}$I'll take my own advice and wrangle this result into a solution.

$$\eqalign{ \Gamma_{ij} &= \frac{\partial(A\otimes A\otimes A\otimes A)}{\partial A_{ij}}\\ &= E_{ij}\otimes A\otimes A\otimes A \;+\; A\otimes E_{ij}\otimes A\otimes A\\ &\quad+\; A\otimes A\otimes E_{ij}\otimes A \;+\;A\otimes A\otimes A\otimes E_{ij}\\ \frac{\partial f}{\partial A_{ij}} &= \Gamma_{ij}\,x \\ \frac{\partial f_k}{\partial A_{ij}} &= e_k^T\,\Gamma_{ij}\,x \\ }$$ where $E_{ij}$ is a matrix of all zeros except for a single $\tt1$ as the $(i,j)$ element and $e_k$ is the standard vector basis.

Note that $E_{ij} = e_ie_j^T.\;$ Also note that $\Gamma$ is the fourth-order tensor whose $(i,j)$ component is the matrix $\,\Gamma_{ij} = \Gamma:E_{ij}\;$ used above. Using the dyadic product $(\star),\,$ the full tensor can be constructed from its components
$$\Gamma = \sum_{i=1}^n\sum_{j=1}^n\;\Gamma_{ij}\star E_{ij}$$

$\def\c#1{\color{red}{#1}}$The Hessian will involve a sixth-order tensor $\cal H$, whose matrix-valued components are $$\eqalign{ {\cal H}_{ijk\ell} &= &\c{E_{ij}}\otimes E_{k\ell}\otimes A\otimes A &+ &\c{E_{ij}}\otimes A\otimes E_{k\ell}\otimes A &+ &\c{E_{ij}}\otimes A\otimes A\otimes E_{k\ell} \\ &+ &E_{k\ell}\otimes \c{E_{ij}}\otimes A\otimes A &+ &A\otimes \c{E_{ij}}\otimes E_{k\ell}\otimes A &+ &A\otimes \c{E_{ij}}\otimes A\otimes E_{k\ell} \\ &+ &E_{k\ell}\otimes A\otimes \c{E_{ij}}\otimes A &+ &A\otimes E_{k\ell}\otimes \c{E_{ij}}\otimes A &+ &A\otimes A\otimes \c{E_{ij}}\otimes E_{k\ell} \\ &+ &E_{k\ell}\otimes A\otimes A\otimes \c{E_{ij}} &+ &A\otimes E_{k\ell}\otimes A\otimes \c{E_{ij}} &+ &A\otimes A\otimes E_{k\ell}\otimes \c{E_{ij}} \\ }$$ The full tensor can be constructed as $$\eqalign{ {\cal H} &= \sum_{i=1}^n\sum_{j=1}^n\sum_{k=1}^n\sum_{\ell=1}^n \;{\cal H}_{ijk\ell}\star E_{ij}\star E_{k\ell} \\ }$$ and the Hessian of $f$ as the product with $x$ $$ \frac{\partial^2 f_p}{\partial A_{ij}\,\partial A_{k\ell}} = e_p^T\,{\cal H}_{ijk\ell}\,x $$

greg
  • 35,825