3

In a euclidean space, we cannot have gradient descent of a function $f$ and still have cycles. i.e. if $x(t)$ is the path traced by the dynamical system given by $\dot x(t)=-\nabla_x f(x)$, for some function $f$, then we cannot have $x(t_0)=x(t_1)\neq x(s)$ for $t_0>s>t_1$.

My question is whether there is an example of an (possibly exotic) space where this doesn't hold. Intuitively I would say it should always hold, but I might not be imagining a weird space where it doesn't.

user56834
  • 12,925
  • Incidentally, I notice that you almost never accept answers. This is not a good behavior on this site. When you feel somebody gave an exhaustive answer, please accept it by clicking on the "check" sign. Thanks. – Giuseppe Negro Jun 10 '19 at 11:13

1 Answers1

4

Let $\langle,\rangle$ be a bilinear form on $\mathbb R^n$, not necessarily positive definite, and define the associated gradient of $f\in C^1$ as the unique vector field $\nabla f(x)$ such that$^{[1]}$ $$ \langle \nabla f(x), v\rangle = \frac{\partial}{\partial \epsilon} f(x+\epsilon v)\Big|_{\epsilon=0}, \qquad \forall v\in \mathbb R^n.$$

If $\langle,\rangle$ is not positive definite, then

the corresponding gradient flow can have cycles (i.e., periodic solutions).

For a concrete example take the bilinear form on $\mathbb R^2$ $$ \langle (x_1, x_2), (y_1, y_2)\rangle:=x_1y_1-x_2y_2.$$ The corresponding gradient is the vector field $$ \nabla f=(\partial_{x_1} f, - \partial_{x_2} f);$$ thus, the associated gradient flow $\dot u =-\nabla u$ is $$ \begin{cases} \partial_t u_1 = -\partial_1f( u_1, u_2),\\ \partial_t u_2= \partial_2 f(u_1,u_2).\end{cases} $$ Taking $f(x_1, x_2)=x_1x_2$ yields $$ \begin{cases} \partial_t u_1 = -u_2,\\ \partial_t u_2= u_1,\end{cases} $$ whose general solution is $(u_1, u_2)=(R\cos (t+\phi), R\sin (t+\phi))$, for arbitrary $R\ge 0$ and $\phi\in\mathbb R$. In particular, all solutions are cycles.

$[1]$. This requires that $\langle,\rangle$ be non-degenerate, which means that the equation $\langle x, y\rangle=0$ can hold for all $y\in\mathbb R^n$ if and only if $x=0$. Notice, moreover, that this is a special case of the standard definition of gradient, in the context of Riemannian and pseudo-Riemannian manifolds. If the bilinear form is the standard dot product, this definition produces the ordinary gradient of vector calculus.

Giuseppe Negro
  • 32,319