1

Suppose we have the function $f(x) = \frac{1}{2} x^T Qx - x^Tb.$ What would be $\nabla f(x)$?

I think I am just getting confused about having both $x^T$ and $x.$ My intuitive answer would be: $\nabla f(x) = \frac{1}{2} Qx - b$ but I'm not sure if that is correct though it seems like it would be since $x^T$ and $x$ represent the same vector.

Michael Hardy
  • 1
juliana paternoster
  • 55
  • 5
  • Here $Q$ is an $n \times n$ matrix. You should always think about the case $n = 1$ to help guess the correct formula for the general case. (This would reveal what's wrong with $\frac12 Qx - b$.) – littleO Jun 05 '21 at 00:24
  • See my edits from proper MathJax usage. In particular $\bigtriangledown f$ looks different from $\nabla f,$ and the latter is standard in this context. – Michael Hardy Jun 05 '21 at 00:32
  • . . . and $f(x) = \frac{1}{2} x^T Qx - x^Tb$ is entirely in MathJax, with just one pair of dollar signs. – Michael Hardy Jun 05 '21 at 00:33

1 Answers1

1

The gradient of $g(x) = x^T A x$ is $g'(x) = (A + A^T) x$ The gradient of $h(x) = x^T b$ relies on the fact that $x^T b$ is a scalar, and a scalar transposed is itself. Therefore, $x^T b = b^T x$. Therefore, the gradient of $h$ is $h'(x) = b$.

One can think of $g(x) = x^T A x$ as analogous to $\tilde{g}(x)=kx^2 = x k x$ when k and $x$ are both scalars. The derivative of $\tilde{g}$ is $2 k x$. But, since matrices are more interesting than scalars, instead of $2 k$ we have $(A + A^T)$. It's at least interesting to note that if $A$ were a scalar, then $A+A^T = 2A$, which satisfies the derivative when working with scalars.

From here, you can use the linearity of the derivative and distribute over the sum to get your answer.

NicNic8
  • 6,951