Derivative of $F(Ax)$

Question

What is the identity for $$ \frac{\partial \mathbf{F}(\mathbf{A}\mathbf{x})}{\partial \mathbf{x}} = ?$$

If $\mathbf{A} \in \mathbb{R}_{mn}$, $\mathbf{x} \in \mathbb{R}_n$, and $\mathbf{F}: \mathbb{R}^m \rightarrow \mathbb{R}^m $ , where $\mathbf{F}(\mathbf{x}) = [f(x_1) \; f(x_2) \; ... \; f(x_m)]^T $ and $ f: \mathbb{R} \rightarrow \mathbb{R} $

I didn't find it here or there.

Just a clarification: is it $dF(Ax)$ of $dG(x)$ with $G(x)=F(Ax)$ uou are asking for? — Julien, Feb 18 '13 at 18:11
@julien it's equal to $dF(Ax)=dG(x)$ with $G(x)=Ax$, A is a matrix and x a vector. — DBS, Feb 18 '13 at 18:17
You mean $G(x)=F(Ax)$. This can't be, these are two different things. Like $f'(2x)$ is not the derivative of $f(2x)$. — Julien, Feb 18 '13 at 18:27
So $F$ is a vector field, $A$ is a linear operator (a matrix), and $x$ is a vector? — Muphrid, Feb 18 '13 at 18:28
Plese consider the remainder of my last comment: $dF(Ax)$ and $dG(x)$ are two different things. — Julien, Feb 18 '13 at 18:33
@julien Suppose you have $y=Ax$, and $F(y)=[sin(y_1) ; sin(y_2) ; ... ; sin(y_m)]$. That's what I wanted.. — DBS, Feb 18 '13 at 18:55
I understand this, but you still haven't answered my question. And you probably did not consider the example $\frac{d}{dx}(f(2x))=2f'(2x)\neq \frac{df}{dx}(2x)=f'(2x)$. — Julien, Feb 18 '13 at 19:05
@julien ah ok.. sorry for the misunderstanding julien, I need $\frac{\partial}{\partial x} [F(Ax))]$, the derivative of everything inside — DBS, Feb 18 '13 at 19:12
Then it is the chain rule. Each of the two functions in the composition $F(Ax)$ is easy to differentiate. — Julien, Feb 18 '13 at 19:16

score 1 · Answer 1 · answered Feb 18 '13 at 18:53

Just use the chain rule. Linear operators are essentially vector fields; they're just linear. This viewpoint makes the application of multivariable calculus very straightforward.

Let $x' = Ax$. Let $\nabla$ be the usual vector derivative (that is, $\partial_x$) and $\nabla'$ be that corresponding to the dimension of $x'$. Let $a$ be a vector, and the chain rule tells us that

$$a \cdot \nabla (F \circ A)(x) = [a \cdot \nabla A(x)] \cdot \nabla' F(x')$$

$A$ is a linear function, so $a \cdot \nabla A(x) = A(a)$, so the result is

$$[A(a) \cdot \nabla'] F(x')$$

You can then evaluate the components by plugging in basis vectors for $a$. In index notation, this yields

$$\sum_{j'} {A_i}^{j'} \partial_{j'} F_{k'}$$

This has two free indices, so the result can be interpreted as a matrix of partial derivatives of the component functions.

does it work for $y=Ax$, and $F(y)=[sin(y_1) ; sin(y_2) ; ... ; sin(y_m)]$ ? — DBS, Feb 18 '13 at 18:58

greg · Answer 2 · 2021-05-28T03:24:38.383

$\def\D{\operatorname{Diag}}\def\p{{\partial}}\def\grad#1#2{\frac{\p #1}{\p #2}}\def\hess#1#2#3{\frac{\p^2 #1}{\p #2\,\p #3^T}}$For typing convenience, introduce the vector variable $$\eqalign{ y &= Ax \quad\implies\quad dy = A\,dx \\ }$$ Denote the scalar function and its derivative as $\big(f,f'\big)\,$ and denote their element-wise application on the new vector variable by the vectors $$\eqalign{f &= f(y) \qquad f' &= f'(y)}$$ It will also be convenient to use an uppercase letter to denote the matrix $$F' = \D(f')$$ Write the differential of the function using the elementwise/Hadamard product, replace the Hadamard product with the diagonal matrix, then perform a change of variables from $y\to x\,$ to find the desired gradient. $$\eqalign{ df &= f'\odot dy \;=\; F'dy \;=\; F'A\,dx \\ \grad{f}{x} &= F'A \\ }$$ In your specific example $$\eqalign{ f &= \sin(Ax) \qquad f' = \cos(Ax) \\ \grad{f}{x} &= \D\left(\cos(Ax)\right)\,A \\ }$$

Derivative of $F(Ax)$

2 Answers2