What is the derivative of Hadamard product of two matrices with respect to one of them? I.e. what is $D(AB)$ with respect to $A$?
Asked
Active
Viewed 3,245 times
3 Answers
6
Consider the 6th-order tensor ${\mathbb M}$, whose components ${\mathbb M}_{ijklmn}$ are unity if $\,(i=k=m)$ and $(j=l=n),\,$ but zero otherwise.
This tensor can be used, along with the Frobenius Product (:), to rewrite the Hadamard Product ($\circ$) between two matrices as
$$\eqalign{
B\circ A &= B:{\mathbb M}:A \cr\cr
}$$
This makes the finding the gradient (a 4th-order tensor) especially simple
$$\eqalign{
F &= A\circ B = B\circ A = B:{\mathbb M}:A \cr\cr
dF &= B:{\mathbb M}:dA \cr\cr
\frac{\partial F}{\partial A} &= B:{\mathbb M} \cr
}$$
greg
- 35,825
3
For the derivative with respect to one component of $A$: $$\frac{\partial}{\partial a_{ij}} (AB)_{kl} = \frac{\partial}{\partial a_{ij}} a_{kl}b_{kl} = b_{kl}\delta_{ik}\delta_{jl}.$$
You'll then have to pick some representation for writing down the full derivative; it is a rank four tensor.
user7530
- 49,280
-
Isn't $b_{kl} \delta_{ik} \delta_{jl}$ just equal to $b_{ij}$? So isn't the derivative of $AB$ with respect to $A$ just equal to $B$? – Joe Apr 08 '20 at 17:43
1
Let $f:A\in M_n\rightarrow A\circ B\in M_n$. $f$ is linear ! Then $Df_A:H\in M_n \rightarrow H\circ B$.
EDIT. @ bordeo , the derivative is a linear application; $H$ is any matrix of $M_n$. In particular, the Taylor formula can be written: $f(A+H)=f(A)+Df_A(H)+\cdots$, that is $(A+H)\circ B=A\circ B+H\circ B+\text{"nothing"}$; "nothing" because $f$ is linear and the terms of degree $\geq 2$ in the Taylor formula are $0$.