Derivative of $f(x) = e^{a^Tx}$

Question

Let $f(x) = e^{a^Tx}$ where $a,x \in \mathbb{R}^N$

Is $\frac {\partial f}{\partial x} = e^{a^Tx}a$ ?

Answer 1

Consider the derivative with respect to $x_i$: $$ \frac{\partial f}{\partial x_i} = \exp\left(\sum_{j \neq i}a_jx_j\right)a_i \exp\left(a_i x_i\right) = a_i\exp\left(\sum_{j}a_jx_j\right) = a_i e^{a^Tx} $$

Since the derivative is most often denoted as the transpose of the gradient, we have $$ Df = a^T e^{a^Tx} \\ \nabla f = ae^{a^Tx} . $$

Answer 2

If $a \in R^N$, $x \in R^N$, we can write $x = (x_1, x_2, . . . , x_N)$, $a = (a_1, a_2, . . . , a_N)$ in the standard coordinate system. Denoting the transpose of $a$ by $a^T$, we then have

$a^Tx = \sum_{i = 1}^{i = N}a_ix_i$,

so that

$f(x) = \exp(\sum_{i=1}^{i = N}a_ix_i)$.

Then for any $x_j$,

$\frac{\partial f}{\partial x_j} = \exp(\sum_{i =1}^{i = N}a_ix_i)a_j$.

Thus

$\nabla f = \exp(\sum_{i = 1}^{i = N}a_ix_i)(a_1, a_2, . . . , a_N) = \exp(\sum_{i = 1}^{i = N}a_ix_i)a$.

This gives the most general derivative of $f(x)$ in the classic sense; I have occasionally seen the notation $\frac{\partial f}{\partial x}$ used to denote $\nabla f$.

An alternative to the above view may be had by considering $Df(x)$ to be the linear map such that

$f(x + h) = f(x) + Df(x)h + e(h)$,

for $h \in R^N$, where $e(h)$ is an error term with the property that $\frac{||e(h)||}{||h||} \to 0$ as $||h|| \to 0$. Then since

$f(x + h) = \exp(\sum_{i = 1}^{i = N}a_i(x_i + h_i)) = \exp(\sum_{i = 1}^{i = N}a_ix_i)\exp(\sum_{i = 1}^{i = N}a_ih_i)$,

and $\exp(\sum_{i = 1}^{i = N}a_ih_i)$ may be expanded in the usual power series

$\exp(\sum_{i = 1}^{i = N}a_ih_i) = 1 + \sum_{i = 1}^{i = N}a_ih_i + \frac{1}{2!}(\sum_{i = 1}^{i = N}a_ih_i)^2 + . . . $,

it is easily seen that

$f(x + h) = \exp(\sum_{i=1}^{i = N}a_ix_i) + \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i + e(h)$ $= f(x) + \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i + e(h)$

showing that

$Df(x)h = \exp(\sum_{i=1}^{i = N}a_ix_i)\sum_{i = 1}^{i = N}a_ih_i = \exp(\sum_{i=1}^{i = N}a_ix_i)a^Th = f(x)a^Th$,

whence

$Df(x) = \exp(\sum_{i=1}^{i = N}a_ix_i)a^T = f(x)a^T$.

I have also occasionally seen $\frac{\partial f}{\partial x}$ used in this context as well.

The difference between $\nabla f$ and $Df$ is, of course, that the former is technically a vector field while the latter is a one-form field (that is, a field of linear functionals on the tangent spaces $T_xR^N$). But in Euclidean space, where forms and vectors are easily identified, the distinction twixt $\nabla f$ and $Df(x)$ is often overlooked.