4

I know that this question has been asked before ($|\nabla f (x)| =1$ implies $f$ linear?) but I struggle to fill in the details of the step by step "proof" given in the answer. What I've tried is here under, but there are still some holes. For example, I can't show $x \cdot u = 0 \implies x \in S$, and overall, I find the proof rather ugly. Such a simple problem ($|\nabla(f)| = 1$) with such a simple solution ($f$ linear), yet so not straightforward) Could anyone please make this more rigorous? Thanks in advance!

--

First, note that the paths of steepest ascent/descent are straight lines. Indeed, these are defined to be the solutions to the ODE $\gamma'(t) = (\nabla f)(\gamma(t)) $, and if we let $t_1, t_2 \in \mathbb{R}$ be arbitrary but different, we can apply the mean value theorem to $t \to f(\gamma(t))$ to obtain $$\frac{|f(\gamma(t_2)) - f(\gamma(t_1))|}{|t_2 - t_1|} = | (f\circ \gamma)'(t_0)| = |(\nabla f)(\gamma(t_0)) \cdot \gamma'(t_0)| = 1.$$ By $K$-Lipschitz continuity of $f$ this implies $$ |\gamma(t_2) - \gamma(t_1)| \geq |t_2 - t_1|.$$ Since $t_2$ and $t_1$ were arbitrary and since $\gamma$ has a constant speed of $1$, this can only be if $\gamma$ is a straight line.

Now let $S$ be the level surface where $f(x) = 0$. Let $p \in S$ be a point and $L$ the line of steepest ascent/descent through $p$. Let $u$ denote the unit vector which determines the direction of $L$, or $u = \nabla f(p)$. Now perform a translation on the coordinate system such that $p=0$ (this changes nothing, we can always retranslate back at the end of the proof). Now, using the mean value theorem again in a similar manner, we can find that $|f(ut)| = |t|$ and by continuity and smoothness, it's either $f(ut) = t$ for all $t\in \mathbb{R}$, or $f(ut) = -t$ for all $t \in \mathbb{R}$. Hence, by $f$ being $1$-Lipschitz, we have $$|tu - s| \geq |t|,$$ for all $s \in S$ and all $t \in \mathbb{R}$. This implies $u \perp s $:

Indeed, let $s \in S$ and write $s = a + b$ where $a \perp u$ and $b \parallel u$ but suppose $b \neq 0$. Write $b = t_b u$ for some $t_b \in \mathbb{R} \setminus \{0\}$. By the Pythagorean theorem and the above, $$(t-t_b)^2 + |a|^2 = |tu - t_b u|^2 + |a|^2 =|tu - a - b|^2 \geq t^2$$ for all $t \in \mathbb{R}.$ This is a contradiction because if we pick $|t|$ large enough, and give $t$ the opposite sign of $t_b$, the left hand side will become greater than the right hand side.

Hence, we can find that $S = \{x \mid x \cdot u = 0\}$ or that $S$ is a plane. With some more work, we can find that all the level surfaces of $f$ are planes parallel to $S$. Depending on whether $f(ut) = t$ or $f(ut) = -t$ was the case earlier, we find $$S + ut = \{x \mid f(x) = t\} \text{ or } \{x \mid f(x) = -t\}.$$ And then, $f(x) = \pm (u_1 x_1 + \ldots + u_n x_n)$, what we wanted to show. (Retranslating the coordinate system yields the constant term in the formula for $f(x).$)

  • 1
    my suggestion is to remove any phrasing that is not strictly mathematical, like "...$L$ the line of steepest descent etc." Instead, write the mathematical characterization of $L$ and make every step of the way formal. In my experience, these type of physics languages hardly ever help the clarity of a pure mathematical proof. – dezdichado May 12 '23 at 19:55
  • @dezdichado Good point. I agree. I believe I've completed the proof now (see answer). – Neckverse Herdman May 13 '23 at 08:19

2 Answers2

2

It happens that we have proved this theorem in a French publication in 1978

'Seules les affinites preservent les lois normales' G. Letac and J. Pradines, Comptes rendus de l'Academie des Sciences de Paris, Serie A, volume 286, pages 399-402.

English title: "Only affinities preserve Gaussian distributions'

  • I believe I have found it! If that is okay, I used it to answer my own question (I had to go slow and translate because my French is quite lacking). Also performed a few more intermediate steps. – Neckverse Herdman May 13 '23 at 08:09
1

Following the (French) proof by the gentleman @Gérard Letac here under (major thanks!), I think I can answer my own question. I assume extra that $f$ is $C^2$, which isn't a problem for what I intend to do. It would be nice to have an extra proof of the part which uses the reference (or maybe fill that last part in with different argument), I'm not that well versed in differential geometry so I don't really understand the theorem which is referenced, but I do understand the result (about principal curvatures).

--

For each $a\in \mathbb{R}^n$, let $M(a) = \{x \in \mathbb{R}^n \mid f(x) = f(a)\}$ be the level surface of $f$ at $a$. We can see that this is a closed $C^1$ submanifold. Further define:

  • $n_a = \nabla f (a)$,
  • $\gamma_a(t)$ the curve which is the (unique) solution to the ODE $\gamma'(t) = \nabla f(\gamma(t))$, $\gamma(0) = a$, of course given by the direct integration of the ODE, and,
  • $g_a(t) = f(a + t n_a)$.

From the hypothesis, we find for each coordinate $x_j$ that $$ 0 = \frac{\partial}{\partial x_j} \left(\sum_{i=1}^n\left( \frac{\partial f}{\partial x_i} \right)^2 \right) = 2 \sum_{i = 1}^n \frac{\partial f}{\partial x_i} \frac{\partial^2 f}{\partial x_i \partial x_j},$$ from which follows that $$\gamma_a ''(t) = \frac{d}{dt} \nabla f(\gamma_a(t)) = \left[\frac{\partial^2 f}{\partial x_j \partial x_i}\right]_{i,j} \cdot \gamma_a'(t) = \left.\sum_{i = 1}^n \frac{\partial f}{\partial x_i} \frac{\partial^2 f}{\partial x_i \partial x_j}\right|_{x\: = \: \gamma_a(t)} = 0. $$ Hence, twice integrating yields $\gamma_a(t) = a + tn_a$. From this, we then find $$g_a'(t) = \nabla f(a+t n_a) \cdot \gamma_a'(t) = \nabla f(a+t n_a) \cdot \nabla f(a+t n_a) = 1,$$ hence, integrating yields $g_a(t) = f(a) + t$.

Now, if $k$ is a principal curvature of $M(a)$ at $a$, then $k/(1-tk) \in \mathbb{R}$ must be the corresponding principal curvature of $M(\gamma_a(t))$ at $\gamma_a(t)$ (Reference 1, p.36). This can only be possible for each $t \in \mathbb{R}$ if $k = 0$ and therefore $M(a)$ is flat at $a$. But we can make the same argument for all $x \in M(a)$ replacing $M(x) = M(a)$ and $x$ with $a$. It follows that $M(a)$ is flat and hence a subset of a hyperplane. Therefore, $x \in M(a) \implies (x-a) \cdot n_a = 0$ and $n_a = n_x$.

From all this, it follows that for $t \in \mathbb{R}$, the level surface $\{y \in \mathbb{R}^n \mid f(y) = f(a) + t\}$ is equal to $M(a) + t n_a = \{x + t n_a \mid f(x) = f(a) \}$, since for all $y \in \mathbb{R}^n$: $$y \in M(a) + t n_a \iff y = x + t n_a, x \in M(a) \iff f(y) = f(a) + t .$$ The latter $\impliedby$ requires some more explanation: suppose $f(y) = f(a) + t$. Define $x = y - t n_a$. We have to show $f(x) = f(a)$. Now, perform all the steps in the proof for $x$, where we replace $a$ by $x$. We hence obtain $n_x, \gamma_x(t) = x + t n_x, g_x$ and $f(x + t n_x) = f(x) + t$. Suppose that $t_0 = f(a) - f(x).$ Then, $x + t_0 n_x \in M(a)$, but since $M(a)$ is a hyperplane with unit normal $n_a$, we obtain $\nabla f(x + t_0 n_x)$ and hence $\gamma_x(t) = n_x$ is also $\gamma_x(t_0) = n_a$. And so, on the one hand, $$f(y) = f(a) + t$$ by assumption, and on the other $$f(y) = f(x + t n_a) = f(x + t n_x) = f(x) + t.$$ This proves $f(x) = f(a)$.

Now we can conclude. For $y$ arbitrary, let $t$ be such that $f(y) = f(a) + t$. By the above, $y = x + t n_a$ for $x$ with $(x-a) \cdot n_a = 0$. Equivalently: $$t = y \cdot n_a - (a + (x-a)) \cdot n_a = (y-a) \cdot n_a,$$ hence, for all $y \in \mathbb{R}^n$ : $ f(y) = f(a) + (y-a) \cdot n_a$, which is what we wanted to show.

Reference 1: N. J. Hicks. Notes on Differential Geometry, Van Nostrand. 1965. A (free, under fair use) 2022 edition can be found here: (https://aareyanmanzoor.github.io/assets/hicks.pdf) but the page number is different: p. 46.

  • 1
    It is a beautuful fact: a local information going to a global result. The lack of singularity is essential since the norm of the gradient of $x\mapsto |x|$ is one but does not exist on $x=0.$ – Gérard Letac May 14 '23 at 22:20
  • @GérardLetac If we allow $f$ to have a singularity at one point $x = a$, are those then the only solutions? Namely, $f$ affine and $f(x) = |x-a|$? How would you go about proving that? Have you also thought about like the similar question $|\nabla(f)| = |x|$? Using the same technique I find that the curves $\gamma$ which follow the gradient are hyperbolas (same argument shows $\gamma'' = \gamma$, from which this easily follows using theory of differential equations). – Neckverse Herdman May 16 '23 at 07:17
  • I suppose it would be quite elegant if these solutions would be quadratic! – Neckverse Herdman May 16 '23 at 07:17
  • I am not an expert in partial differential equations. The equation $|f'(x)|^2=F(x)$ is called the eikonal equation in optics and has an immense literature. – Gérard Letac May 16 '23 at 16:10