1

I'm not familiar with derivations of equations involving vectors and matrices. Given

$$f(x)=c^Tx + y^TAx$$

with $y \in \mathbb{R}^d, A \in \mathbb{R}^{d\times n}, x \in \mathbb{R}^n, c \in \mathbb{R}^n$.

What is the derivative of $f(x)$? Somehow $c^T + y^TA$ can not be right since the second term yields a matrix and no scalar as in $f(x)$.

A.Γ.
  • 29,518
user2820379
  • 313
  • 1
  • 2
  • 11

1 Answers1

2

The function is linear in $x$ $$ f(x)=(\underbrace{c+A^Ty}_{=d})^Tx=d^Tx=d_1x_1+d_2x_2+\ldots+d_nx_n. $$ The derivative of $f(x)$ for $f\colon\mathbb{R}^n\to \mathbb{R}$ is the gradient which is defined as a vector of partial derivatives $$ \nabla f(x)=\left[\matrix{\frac{\partial}{\partial x_1}f\\\frac{\partial}{\partial x_2}f\\ \vdots\\\frac{\partial}{\partial x_n}f}\right]= \left[\matrix{d_1\\d_2\\\vdots\\d_n}\right]=d=c+A^Ty. $$ P.S. You did right, but it is common to think of the gradient as a column vector.

A.Γ.
  • 29,518
  • Isn't it actually usual that Jacobian matrix is $(\partial_jf_i)_{ij}$? – Ennar Aug 16 '15 at 09:16
  • Yes, in this sense the gradient becomes the transposed Jacobian. One can say that it is more natural to define gradient=Jacobian, i.e. a row vector, but it is convenient to think of $x$ and $\nabla f$ to be the elements of the same $\mathbb{R}^n$ as columns. It is just an agreement. At least I have seen it to be defined as a column more often than otherwise. – A.Γ. Aug 16 '15 at 09:29