1

Suppose I have a matrix $f = \begin{bmatrix}x_1 & x_2 \\ 1 & 0\end{bmatrix}$. I would like to compute $\frac{dx'fx}{dx}$ with $x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$.

One way is to first compute $x'fx = x_1^3+x_1x_2+x_1x_2^2$, then differentiate it: $\frac{dx'fx}{dx} = \begin{bmatrix}3x_1^2+x_2+x_2^2 \\ x_1+2x_1x_2\end{bmatrix}$.

The other way is to rewrite $\frac{dx'fx}{dx} = fx + vec\{x'f_xx\}$ using product rule. Here vec is a operator to stack the scalar. Then I have to compute $f_x$. This is where I stuck.

There are two issues here, note that $f_x$ is a matrix to vector derivative, hence we get a tensor with each element a 2 by 2 matrix. The order matters how this tensor is formed, i.e. each element could be $\frac{df}{dx_i}$ or $\frac{df_i}{dx}$, which should I choose for my purpose?

Through some computations, I note that none of them leads to the correct answer simply becasue $fx = \begin{bmatrix}x_1^2+x_2^2 \\ x_1\end{bmatrix}$, so $x_2$ in the first row can never be achieved by $vec\{x'f_xx\}$.

Can anyone please tell me what I am wrong here? Thanks in advance.

Tianyu
  • 11
  • 1
    You can write the matrix as $F=(e_1x^T+e_2e_1^T).;$ Then the scalar function becomes $$(x^TFx) = x^Te_1x^Tx + x^Te_2e_1^Tx$$ which you can differentiate term-by-term $$\frac{\partial(x^TFx)}{\partial x} = 2(x^Te_1)x + (x^Tx)e_1 + (x^Te_2)e_1 + (x^Te_1)e_2$$ – greg Jul 20 '21 at 21:39
  • Thanks. This is a neat trick for solving this particular example. However, the question could be more general, i.e. f contains some nonlinear function of x. Actually, the point here is I wonder how this problem can be approached directly through the product rule. – Tianyu Jul 21 '21 at 14:01
  • 1
    You have vectorized the function incorrectly, it should read $$\eqalign{ x^TFx &= {\rm vec}(x^TFx) \ &= (x\otimes x)^T{\rm vec}(F) \ &= (x\otimes x)^Tf \ d_k(x^TFx) &= (x\otimes x)^Td_kf + (d_kx\otimes x+x\otimes d_kx)^Tf \ }$$ where I've used the abbreviation $;d_k\to \frac{\partial}{\partial x_k}$ and $d_kx = e_k\qquad$ – greg Jul 21 '21 at 15:47
  • I'm not vectorizing anything. The vec operator is used to stack the tensor. What I did is just finding the derivative $\frac{dx'fx}{dx} = fx + vec{x'f_xx} $ using the product rule. Is it wrong? – Tianyu Jul 21 '21 at 19:22

0 Answers0