0

Let $F(X)=XX^\top$ where $X$ is a $n \times m$ matrix. What is $\frac{\partial F(X)}{\partial X}$ ?

$$ \begin{align} \frac{\partial F(X)}{\partial X}=\lim_{h \to 0}\frac{F(X+hZ)-F(X)}{hZ} & = \lim_{h \to 0}\frac{XX^\top + hXZ^\top +hZX^\top + h^2ZZ^\top - XX^\top}{hZ} \\ & = \lim_{h \to 0}\frac{hXZ^\top+hZX^\top+h^2ZZ^\top}{hZ} \\ \end{align} $$ Observe that $\lim_{h \to 0}\frac{h^2ZZ^\top}{hZ} = 0$. $$ \lim_{h \to 0}\frac{F(X+hZ)-F(X)}{hZ} = \lim_{h \to 0}\frac{hXZ^\top+hZX^\top}{hZ} = \frac{XZ^\top+ZX^\top}{Z} $$ Now, by convention, let's say that $\frac F X = \frac{vec(F)}{vec(X)^\top}$ and $\frac{vec(X)}{vec(X)^\top}=I \otimes I$.

So we have : $$ \begin{align} \frac{XZ^\top+ZX^\top}{Z} & = \frac{(I \otimes X)vec(Z^\top)+(X \otimes I)vec(Z)}{vec(Z)^\top} \\ & = \frac{\left[(I\otimes X)(I\boxtimes I) + (X\otimes I)\right]vec(Z)}{vec(Z)^\top} \\ \frac{\partial F(X)}{\partial X}& = I \boxtimes X + X \otimes I \end{align} $$

Is it correct ?

davcha
  • 1,745

1 Answers1

0

This helps a lot : http://www.janmagnus.nl/misc/mdc2007-3rdedition

Let $F:\mathbb R^{n \times m} \mapsto \mathbb R^{p \times q}$ and $X \in \mathbb R^{n \times m}$. Then, $$ DF(X)=\sum_{i=1}^p\sum_{j=1}^q vec\left(\frac{\partial F(X)}{\partial x_{ij}}\right)vec(E_{ij})^\top $$ And, $$ \frac{\partial F(X)}{\partial x_{ij}} = \lim_{h \to 0}\frac{F(X+hE_{ij})-F(X)}{h} $$ From there, we have \begin{align} \tag{1}DF(X) & = \sum_{i=1}^p \sum_{j=1}^q vec(Xe_je_i^\top + e_ie_j^\top X^\top)vec(e_ie_j^\top)^\top \\ & = \sum\sum\left[(I \otimes X)vec(e_je_i^\top)+(X \otimes I)vec(e_ie_j^\top) \right]vec(e_ie_j^\top)^\top \\ & = \sum\sum\left[(I \otimes X)T_{q,p}vec(e_ie_j^\top)+(X \otimes I)vec(e_ie_j^\top) \right]vec(e_ie_j^\top)^\top \\ \end{align} Where $T_{q,p}$ is the permutation matrix. \begin{align} DF(X) & = ((I \otimes X)T_{q,p}+X \otimes I)\left[\sum\sum vec(e_ie_j^\top)vec(e_ie_j^\top)^\top\right] \\ & = (I \otimes X)T_{q,p}+X \otimes I \end{align}

Is it correct, now ?

davcha
  • 1,745