2

I read the following claim in the book, P19 Eq.(3.9).

For a smooth function $f:\mathcal{E} \rightarrow R$, where $\mathcal{E}$ is a linear space. $Df(x): \mathcal{E}\rightarrow R$ is the differential of $f$ at $x$, that is, it is the linear map defined by: $$Df(x)[v]=\lim\limits_{t\rightarrow 0} \frac{f(x+tv)-f(x)}{t}.$$

  1. Is this the standard definition of differential?
  2. Is the claim "$Df(x)$ is a linear map" inferred from the limit expression?
Chuan Huang
  • 67
  • 3
    Your $Df(x)$ may no be linear, not even if $\varepsilon=\Bbb R^2$. I don't understand what the scope of the discussion in your textbook is, but in the normed space $\Bbb R^2$ it's entirely possible to have some $f:\Bbb R^2\to\Bbb R$ such that $v\mapsto \frac{f(x+tv)-f(x)}{t}$ is a well-defined linear map, but which still doesn't satisfy the standard definition of "differential of $f$ at $x$", id est "a linear map $L:\Bbb R^2\to \Bbb R$ such that $\lim_{\lVert v\rVert\to 0}\frac{f(x+v)-f(x)-Lv}{\lVert v\rVert}=0$. – Sassatelli Giulio Mar 02 '24 at 19:55
  • @SassatelliGiulio This definition appeared on page 19 (Eq. (3.9)) in the book available at https://www.nicolasboumal.net/book/#book. – Chuan Huang Mar 02 '24 at 20:02
  • What horrendous notation, using $\varepsilon$ for a linear space. You need to assume that $f$ is differentiable in order to use the directional derivative formula for the derivative. There are non-differentiable functions, for example, with the property that every directional derivative at $0$ is $0$. And standard examples where the directional derivative is not a linear function at all. – Ted Shifrin Mar 02 '24 at 20:09
  • 1
    You forgot to specify that $f$ is smooth (and that $\varepsilon$ is a finite-dimensional vector space, which helps the discussion). Then it's just part of the definition of differentiability (the one that I've written). – Sassatelli Giulio Mar 02 '24 at 20:12
  • @TedShifrin It's largely the author's fault for using mathcal E (i.e. $\mathcal E$) instead of something like $E$. – Sassatelli Giulio Mar 02 '24 at 20:14
  • The point is that the author is assuming that $f$ is smooth. All you need is differentiability at the point. Then it is standard (see my text or YouTube lectures) that $Df_a(v)$ is in fact the directional derivative of $f$ at $a$ in direction $v$. You can, if you insist, think of this as a special case of the chain rule, but it's just immediate from the definition of the derivative. – Ted Shifrin Mar 02 '24 at 20:19
  • I still failed to understand $Df(x)$ being the linear map, is it by definition that it is a linear map or can it be verified using the directional derivative definition $Df(x)[v]=\lim\limits_{t\rightarrow 0} \frac{f(x+tv)-f(x)}{t}$? @TedShifrin – Chuan Huang Mar 02 '24 at 20:38
  • 1
    The definition of differentiability is crucial. As we've commented, without that hypothesis, the directional derivative will not be linear in $v$. Maybe this lecture will help if you take the time. – Ted Shifrin Mar 02 '24 at 20:49
  • 1
    To reiterate Sassatelli's and Ted's comments from a different angle: The question reads (emphasis added), "[$Df(x)$] is the linear map defined by...." The difference quotient expression tells us how to evaluate the linear map $Df(x)$ on a vector $v$. But the question asked here appears to be, "Assuming the difference quotient limit exists for all $v$, how do we establish $Df(x)$ is linear?" Since linearity does not follow from existence of the right-hand limit for all $v$, the requested implication cannot be established. – Andrew D. Hwang Mar 02 '24 at 22:07
  • @AndrewD.Hwang I agree with you. I think the differential cannot be defined by that difference quotient expression, it is because of the differentiability definition that makes it a linear map. – Chuan Huang Mar 02 '24 at 22:20
  • Maybe I'm confused what the question is, then. Is the concern that "defining" $Df(x)[v]$ by the limit of difference quotients is somehow circular or otherwise unfounded? – Andrew D. Hwang Mar 02 '24 at 22:31
  • 2
    @AndrewD.Hwang I initially thought $Df(x)$ is implicitly defined by the limit of difference quotients in the book, so I tried to verify the linearity by using the limit of the difference quotient, but it did not work. Now I understand $Df(x)$ has a standard definition that it is a linear map. – Chuan Huang Mar 02 '24 at 23:07

2 Answers2

4

The differential $Df(x)$ of $f$ at $x$ is linear by definition (whenever it exists). The limit you wrote is not the definition of $Df(x)(v)$ but of the directional derivative of $f$ at $x$ in the direction of $v$.

Bad news:

Good news:

  • If $Df(x)$ exists, then the directional derivative of $f$ at $x$ in the direction of $v$ exists and is equal to $Df(x)(v)$. It is then de facto a linear function of $v$.
Anne Bauval
  • 34,650
-1

Fix some $x \in \text{Dom}(f)$.

Then as you have defined

$$Df(x)[v] = \lim_{t \rightarrow 0} \frac{f(x+tv)-f(x)}{t}$$

and from this we have

$$Df(x)[v] = v \lim_{t \rightarrow 0} \frac{f(x+tv)-f(x)}{tv}$$

And then let $\Delta x = tv$. As $t \rightarrow 0$, $\Delta x \rightarrow 0$. So we have

$$Df(x)[v] = v \lim_{\Delta x \rightarrow 0} \frac{f(x+\Delta x)-f(x)}{\Delta x} = v f'(x)$$

As we can see, for each $x \in \text{Dom}(f)$, $Df(x)[v] = v f'(x)$ is a linear map, because if $x$ is fixed, then $f'(x)$ is a constant or infinity.

Hope this can help you.

ZYX
  • 1,154
  • 1
    Thanks, but I thought in general $v$ is not a scalar, then that something divided by $tv$ does not make any sense. – Chuan Huang Mar 02 '24 at 21:23
  • @ChuanHuang Since you write $Df(x)[v] = \lim_{t \rightarrow 0} \frac{f(x+tv)-f(x)}{t}$, it automatically means everything in this expression is scalar, because variables as vectors cannot be written in this form. – ZYX Mar 02 '24 at 21:25
  • 2
    $t$ is a scalar, but $x$, $v$ can be vectors. This is the directional derivative definition. @ZYX – Chuan Huang Mar 02 '24 at 21:28
  • @ChuanHuang Consider $f$ as a function of multiple variables and rewrite the linear transformation in the form of matrix, and remember to write $f'(x)$ as $n \times m$ Jacobian matrix. – ZYX Mar 02 '24 at 21:38
  • 1
    No, even you write $f^{\prime}$ as $n\times m$ Jocabian matrix, $v f'(x)$ is not the directional derivative, it should be $f'(x) v$. In cases where $x$ and $v$ are matrices, $f^{\prime}$ will not be even a matrix. I need to consider the general linear space, not limited to column vectors. – Chuan Huang Mar 02 '24 at 21:49