2

Suppose $m > 1$. Let $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ be a smooth map. Consider $f + Ax$ for $A \in \mathrm{Mat}_{m\times n}$. Define $F: \mathbb{R}^n \times \mathrm{Mat}_{m\times n} \rightarrow \mathrm{Mat}_{m\times n}$ by $F(x,A) = df_x + A$.

I am really grateful to @Ross B. 's answer on the question The derivative of a linear transformation, $DF$ would be a rank-3 tensor with elements

$$ (DF)_{i,j,k} = \frac{\partial^2 f_i}{\partial x_j \partial x_k} $$

Some authors also define matrix-by-vector and matrix-by-matrix derivatives differently be considering $m \times n$ matricies as vectors in $\mathbb{R}^{mn}$ and "stacking" the resulting partial derivatives.

$dF$ maps from $m \times n \times n$ to $m \times n$?

Thank you very much.

Community
  • 1
1LiterTears
  • 4,572

1 Answers1

1

Here it pays to think in terms of (spaces of) linear transformations. In general, if $T : V \to W$ is differentiable, for $V$ and $W$ finite-dimensional real inner product spaces, then $DT : V \to L(V,W)$. Thus, in this case:

  1. By construction, $F : \mathbb{R}^n \times M_{m \times n}(\mathbb{R}) \to M_{m \times n}(\mathbb{R})$.
  2. Thus, $DF : \mathbb{R}^n \times M_{m \times n}(\mathbb{R}) \to L(\mathbb{R}^n \times M_{m \times n}(\mathbb{R}), M_{m \times n}(\mathbb{R}))$.
  3. In particular, if you identify $\mathbb{R}^n \times M_{m \times n}(\mathbb{R}) \cong \mathbb{R}^{n+mn}$ and $M_{m \times n}(\mathbb{R}^n) \cong \mathbb{R}^{mn}$, then you can identify $DF$ as a map $$DF : \mathbb{R}^{n+mn} \to L(\mathbb{R}^{n+mm},\mathbb{R}^{mn}) \cong M_{mn \times (n+mn)}(\mathbb{R}).$$
Branimir Ćaćić
  • 13,684
  • Thanks Branimir. I see that $\mathbb{R}^n$ has dimension $n$ and $M_{m \times n}(\mathbb{R})$ has dimension $mn$. But we differed from here: I thought then the cross has dimension $n \times mn$ instead of +. Could you please point out why I am wrong here? Thanks! – 1LiterTears Jun 12 '13 at 17:49
  • 1
    In general, if $V$ and $W$ are finite-dimensional, then $V \times W = {(v,w) \mid v \in V, ; w \in W }$ has dimension $\operatorname{dim} V + \operatorname{dim} W$, e.g., for $\mathbb{R}^2 = \mathbb{R} \times \mathbb{R}$, $\operatorname{dim} \mathbb{R}^2 = 2 = \operatorname{dim} \mathbb{R} + \operatorname{dim} \mathbb{R}$. – Branimir Ćaćić Jun 12 '13 at 18:49
  • Oh, right, - thanks! – 1LiterTears Jun 12 '13 at 19:42