1

$\frac{d(x^Ta)}{dx} = \frac{d(a^Tx)}{dx} = a^T$

I was confused by this simple formula for a few weeks.

I thought $x^Ta$ is an scalar, and it's derivative respect to a column vector should be an vector, e.g. $a$ instead of $a^T$.

am I missing something?

Thank you !!

Long
  • 269
  • I think it has to do with the fact that the scalar product is defined as $\square \cdot \triangle = \square^\intercal \triangle$ – John Alexiou Jan 05 '16 at 16:53
  • If $ f:\mathbb R^n \to \mathbb R^m $, then $ f'(x)$ is an $ m \times n$ matrix. – littleO Jan 05 '16 at 17:45
  • @ja72 I notice one paradox: could you please explain how this happens? Thank you! $\mathbf{a^T} \frac{d\mathbf{x}}{d\mathbf{x}} = \mathbf{a^T}$ \

    $ \frac{d\mathbf{x^T}}{d\mathbf{x}} \mathbf{a} = \mathbf{a} $

     – Long Jan 06 '16 at 09:59
  • @littleO you mean the Jacobian Matrix? why it's a row vector when the gradient is a column vector? Thank you ! – Long Jan 06 '16 at 10:02
  • @Long replace all $\square^\intercal$ with $\square \cdot$ and see if it makes sense. The transpose is part of an inner product in vector space. – John Alexiou Jan 06 '16 at 13:37
  • @Long If I've answered your question, make sure you remember to click the checkmark left of my answer (it'll turn green once you do). If not, what should I expand on? –  Jan 07 '16 at 04:02
  • @Bye_World So Sorry!!!! This was my first question here, I did not realize that checkmark. Thank you!!!! – Long Jan 07 '16 at 17:43

1 Answers1

2

Here's the definition of the derivative of a function $y: \Bbb R^n \to \Bbb R$ wrt the column vector $\mathbf x$. $$\frac{dy(\mathbf x)}{d\mathbf x} := \pmatrix{\frac{\partial y(\mathbf x)}{\partial x_1}, \dots, \frac{\partial y(\mathbf x)}{\partial x_n}}$$ Notice that this is a row vector (by definition).

$y(\mathbf x) = \mathbf a^T\mathbf x$ is a scalar function so the result of differentiating it wrt $\mathbf x$ must be some row vector. Now I assume you can prove that $\frac{d\mathbf a^T\mathbf x}{d\mathbf x} = \mathbf a^T = \frac{d\mathbf x^T\mathbf a}{d\mathbf x}$ in this case?

  • Thank you so much! I have one more question about this definition. what was the gradient of this y(x) w.r.t x? was it a column vector? – Long Jan 06 '16 at 09:54
  • It has the same components. But whether it's represented by a row or column vector is not entirely standardized. Different authors choose one or the other. –  Jan 06 '16 at 11:57