An equation about the 2nd order Fréchet derivative

Question

This is the book I'm reading: https://link.springer.com/content/pdf/10.1007/978-0-387-68407-9.pdf?pdf=button

Definition 1.7. The directional derivative of $f$ at $x \in U$ along the direction $d \in \mathbb{R}^{n}$ is $f'(x; d) := \lim_{t \downarrow 0} \frac{f(x + td) - f(x)}{t}$, provided the RHS exists.

Definition 1.9. The function $f: U \in \mathbb{R}^{n} \xrightarrow{} \mathbb{R}^{m}$ is called Fréchet differentiable at $x \in U$ is there exists a linear map $B: \mathbb{R}^{n} \xrightarrow{} \mathbb{R}^{m}$ s.t. $\lim_{\lVert h \rVert \xrightarrow{}0} \frac{\lVert F(x + h) - F(x) - Bh \rVert}{\lVert h \rVert} = 0.$ The map $B$, denoted by $DF(x)$, is called the Fréchet derivative of $F$.

Definition 1.20. Let $f$ from $U$ into $\mathbb{R}^{m}$ be a map on an open set $U \in \mathbb{R}^{n}$. We call $f$ twice Fréchet differentiable on $U$ if both $f$ and $Df$ are Fréchet differentiable on $U$, and denote by $D^{2}f := D(Df)$ the second derivative of $f$. The answer here may be helpful: What does it mean to say something is twice Fréchet differentiable?

If $ a \in U$ and $u, v \in \mathbb{R}^{n}$, then we denote by $D^{2}f(a)[u, v]$ the directional derivative of the function $h(a) := f'(a; u)$ along the direction $v$, that is, $D^{2}f(a)[u, v] := h'(a; v)$.

Theorem 1.21 says if $f: U \xrightarrow{} \mathbb{R}$ is twice Fréchet differentiable on the open set $U \in \mathbb{R}^{n}$, then $D^{2}f(a)$ is a symmetric bilinear form for all $a \in U$, that is, $D^{2}f(a)[u, v] = D^{2}f(a)[v, u]$ for all $u. v \in \mathbb{R}^{n}$.

Here is the part I don't understand in the proof of 1.21:

Since $Df$ is Fréchet differentiable, we have

$Df(a + u + tv)(v) - Df(a)(v) - D^{2}f(a)[v, u+tv] = o(\lVert v \rVert \cdot \lVert u+tv \rVert) \leq o((\lVert v \rVert + \lVert u \rVert)^{2})$.

Could anyone explain the equation and the inequality here? Thanks for any help!

Answer 1

The second derivative is a map $DDf:U\to L(\mathbb R^n,L(\mathbb R^n,\mathbb R^m))$ which satisfies $$\|Df(a+h)-Df(a)-DDf(a)(h)\|_{L(\mathbb R^n,\mathbb R^m)} =o(\|h\|)$$ where the norm on the left hand side is the operator norm (or just any norm on $L(\mathbb R^n,\mathbb R^m)$ because on finite dimensional spaces all norms are equivalent). Apparently you then use $h=u+tv$ and evaluate in $v$. Thus, the expression $DDf(a)[v,h]$ should mean first the evaluation in $h$ and then in $v$, i.e., $DDf(a)(h)(v)$. Denoting the difference in the displayed fomula by $S(h)$, the inequality then follows from $$\|S(h)(v)\|\le \|v\| \|S(h)\|_{L(\mathbb R^n,\mathbb R^m)}=o(\|v\|\|h\|).$$ The second inequality is just the triangle inequality together with something elementary like $a(b+a)\le (a+b)^2$ (it seems that parameter $t$ comes from some fixed bounded interval).