What is the derivative of this operation
$A=Trace [(1U-W)\circ(1U-W)]$ with respect to $U$, when $\circ$ represents an element wise (hadamard) product. $1$ is a matrix with same size as $U$ where all elements are $1$.
Asked
Active
Viewed 75 times
1
-
What have you done so far? – TZakrevskiy Jan 15 '14 at 11:11
-
I have tried to convert hadamard product to normal matrix product. – Neshat Jan 15 '14 at 11:13
-
And $U$ and $W$ are quadratic of the same size, right? – martini Jan 15 '14 at 13:42
-
Yes, they are same size – Neshat Jan 15 '14 at 13:48
1 Answers
0
We have for $U, H \in \def\Mat{\mathop{\rm Mat}\nolimits}\Mat_n(\def\R{\mathbb R}\R)$ that$\def\tr{\mathop{\rm Trace}}$ \begin{align*} A(U+H) &= \tr\bigl([1(U+H) - W]\circ [1(U+H)-W]\bigr)\\ &= \tr\bigl((1U - W + 1H)\circ (1U - W + 1H)\bigr)\\ &= \tr\bigl((1U - W)\circ(1U- W) + 2\cdot 1H\circ(1U-W) + 1H \circ 1H\bigr)\\ &= A(U) + 2 \tr \bigl(1H \circ (1U-W)\bigr) + \tr(1H \circ 1H) \end{align*} Now $\def\abs#1{\left|#1\right|}\def\norm#1{\left\|#1\right\|}$ $\tr(1H \circ 1H)$ is a finite sum of two-factor-products of entries of $H$, hence $\abs{\tr(1H \circ 1H)} \le C\norm{H}^2$ for some ($n$-dependent) constant $C$. So $\tr(1H \circ 1H) = o(\norm H)$ and as $H \mapsto 2\tr(1H \circ (1U-W))$ is linear in $H$, this is the derivative, giving $$ A'(U)H = 2\tr\bigl(1H \circ (1U- W)\bigr) $$
martini
- 84,101
-
What is H? I dont have any H in my question. I would like to computze $\frac{\delta A}{\delta U}$ – Neshat Jan 15 '14 at 14:14
-
The definition of derivative I used is the following: $A$ is differentiable in $U$ iff there is a linear $A'(U) \colon \Mat_n(\R) \to \R$ such that $$ A(U+H) = A(U) + A'(U)H + o(\norm H), \quad H \to 0 $$ Then $A'(U)$ is called the derivative of $A$ in $U$. What definition do you know? – martini Jan 15 '14 at 14:20