differentiation with respect to vector

Question

Let $x \in \mathbb{R}^{n}$ a vector, and $A(x) \in \mathbb{R}^{n \times n}$ a symmetric matrix depending on $x$ I need to find a way to differentiate $p(x) = x^t A(x) x \in $ with respect to $x$. This should be a 1 by $n$ vector.

I tried the following \begin{align} \frac{d p(x)}{d x} & = \frac{d x^t A(x) x }{d x} \\ & = \frac{d x^t}{d x} A(x) x + x^t \frac{d A(x) x }{d x} \end{align} But I think this is already incorrect because the first term can't be an 1 by $n$ vector anymore. Does anyone know how to do this?

The derivative of such functions from $\Bbb R^n$ to $\Bbb R$ is usually interpreted as being the gradient $\mathrm{grad}; p$. — M. Winter, May 05 '17 at 08:51
@M.Winter The gradient takes scalar functions as inputs, not vectors. OP, see this. — Bobson Dugnutt, May 05 '17 at 09:15
@Lovsovs: That's probably what he meant. The derivative of mentioned function ( which is a scalar valued function ). Not that the gradient takes us to $\mathbb R$, but the derivative of a function that takes us to $\mathbb R$. — mathreadler, May 05 '17 at 09:17
Also the gradient can in fact be computed for a vector though the resulting object is more complicated — Triatticus, May 05 '17 at 09:22
@Triatticus: Yes, you can do that and get a "matrix" in some sense. Just add one index. Like a tensor outer product with nabla. — mathreadler, May 05 '17 at 09:24
In index notation, the gradient is $$\frac{\partial p}{\partial x_k}=2A_{ki}x_i+x_ix_j\Bigg(\frac{\partial A_{ij}}{\partial x_k}\Bigg)$$ — lynn, May 05 '17 at 14:07

score 1 · Accepted Answer · answered May 05 '17 at 09:35

The derivative of such functions from $\Bbb R^n$ to $\Bbb R$ is usually interpreted as being the gradient $\mathrm{grad}\; p$. When rewriting $p$ in components

$$x^\top A(x)x=\sum_{i,j}x_ix_jA_{ij}(x)$$

you can apply the gradients definition immediately. Let $\partial_k$ be the partial derivative w.r.t. the $k$-th component of $x$, then

\begin{align} \partial_k p(x) &=\sum_j(x_jA_{kj}(x)+x_kx_j[\partial_kA_{kj}(x)])+\sum_{i\not=k,j}x_ix_j[\partial_kA_{ij}(x)]\\ &=\sum_jx_jA_{kj}(x)+\sum_{i,j}x_ix_j[\partial_kA_{ij}(x)]\\ &=e_k^\top A(x)x+x^\top [\partial_kA(x)] x. \end{align}

Take these together to a vector to obtain

$$\text{grad }p(x) = A(x)x+(\partial_1A(x)x \;\cdots\; \partial_nA(x)x)^\top x,$$

where in the second term there is this matrix with rows $\partial_kA(x)x$ (after transposing).

ok, it seems to be correct. Thank you! – Koen May 06 '17 at 18:53 — Koen, May 06 '17 at 18:53

differentiation with respect to vector

1 Answers1