1

Let $f({\bf x})\equiv f(x_1,x_2,\ldots,x_n)$ is a function of $n$ real variables ${\bf x}=(x_1,x_2,\ldots,x_n)$. The Taylor expansion of $f({\bf x})$ is about the point ${\bf x}^0=(x_1^0,x_2^0,\ldots,x_n^0)$, reads, $$f({\bf x})=f({\bf x}^0)+\sum_i\left(\frac{\partial f}{\partial x_i}\right)_{{\bf x}={\bf x}^0}(x_i-x_i^0)+\frac{1}{2!}\sum_i\sum_j\left(\frac{\partial^2 f}{\partial x_i\partial x_j}\right)_{{\bf x}={\bf x}^0}(x_i-x_i^0)(x_j-x_j^0)+\ldots$$ If ${\bf x}^0$ is a local minimum, $$\partial f/\partial x_i=0, \forall i .$$ This is true.

However, is it also true that at ${\bf x}^0$, $$H_{ij}\equiv\left(\frac{\partial^2 f}{\partial x_i\partial x_j}\right)_{{\bf x}={\bf x}^0}\geq 0$$ as claimed here in this Quantum Field Theory Textbook by Lewis Ryder (Eq. 8.24)? See here. If so, can you offer proof of this?

SRS
  • 1,452
  • 1
    The Hessian is positive semi definite. That is what is being referred to in your text, albeit in confusing notation/prose. Also, you probably want to use linear algebra: $f(x) = f(x^0)+\nabla f(x^0)^T(x-x^0)+\frac{1}{2}(x-x_0)^T \nabla^2 f(x^0)(x-x^0)+o(|x-x_0|^2)$, from which it should be clear why the Hessian needs to be positive semi definite. – Andrew Aug 02 '23 at 15:13
  • @AndrewZhang Isn't the condition for a local minimum equivalent to a positive definite Hessian instead of a positive semi-definite Hessian? https://en.wikipedia.org/wiki/Hessian_matrix#Second-derivative_test – SRS Aug 02 '23 at 15:29
  • No. (Filler words to reach character minimum) – Andrew Aug 02 '23 at 15:34
  • 1
    @AndrewZhang you can use the symbols { and } inside dollar signs and they won't print out. – William M. Aug 02 '23 at 15:35
  • No. In one variable, consider $f[x)=x^4$. This generalizes in an obvious way to $n$ variables. – Ted Shifrin Aug 02 '23 at 17:32

2 Answers2

1

You can see the positivity from taking a limit. Note that for a function $g$ with second derivative at $0$ we have

$$g''(0) = \lim_{t\to 0} \frac{g(t) + g(-t) - 2 g(0)}{t^2}$$

and so, if moreover $0$ is a local minimum, we have $g''(0) \ge 0$.

Now take the case of your problem. Let $h\in \mathbb{R}^n$. The functions

$$t \mapsto g(t) \colon = f(x_0 + t h)$$

has a minimum at $t=0$, so $g''(0)\ge 0$. But we have

$$g''(0) = \sum_{i,j} \frac{\partial f}{\partial x_i \partial x_j}_{x=x_0} h_i h_j $$

orangeskid
  • 53,909
  • From RHB book, the condition for a local minimum is $\Delta f>0$ i.e. $\sum_{ij}M_{ij}\Delta x_i\Delta x_j>0$. Is it wrong? See here: https://math.stackexchange.com/questions/4746544/did-i-spot-a-mistake-in-riley-hobson-bence – SRS Aug 02 '23 at 17:51
  • @SRS: It is a sufficient condition. It is not necessary, as other have pointed out with for e.g. $f(x_1,x_2) = x_1^4 + x_2^4$. Here all the partial derivative of order $\le 3$ at $(0,0)$ equal $0$. – orangeskid Aug 02 '23 at 18:54
  • 1
    @SRS: However, in practice such situations almost never happen, so you can pretend that the statement is exactly right. Correcting Math "errors" in Physics books is not worth the time :-) Cheers! – orangeskid Aug 02 '23 at 19:00
  • So at a local minimum is $\Delta f=\sum_{ij}M_{ij}\Delta x_i \Delta x_j\geq 0$ and not necessaility $\Delta f=\sum_{ij}M_{ij}\Delta x_i \Delta x_j>0$. Did I get that right? @orangeskid – SRS Aug 02 '23 at 19:35
  • @SRS Yes, correct :-) – orangeskid Aug 03 '23 at 02:07
0

Taylor theorem assert that $$ f(x + h) = f(x) + f'(x) \cdot h + f''(x) \cdot (h,h) + o(\|h\|^2), $$ this being true for $f$ twice differentiable in a neighbourhood of $x,$ everything in this equation taking place in normed spaces. Then, for any vector $v,$ let $h = \lambda v.$ Suppose $f(x) = \max f(B_x)$ and that $f(y) < f(x)$ for all other $y \in B_x,$ where $B_x$ is some ball centred at $x.$ Then, $f'(x) = 0$ as it is well-known, and $$ f(x + h) - f(x) = \lambda^2 f''(x) \cdot (v,v) + o(\lambda^2), $$ the right hand side is $< 0$ and of the form $c \lambda^2 + \lambda^2 o(1).$ If $\lambda$ is small enough, the sign of this will be that sign of $c = f''(x) \cdot (v,v),$ assuming $c \neq 0,$ therefore $f''(x) \cdot (v,v) < 0$ assuming $c \neq 0.$ Thus, $f''(x)$ is negative semidefinite (i.e., a quadratic opening down).

William M.
  • 7,532
  • I may be wrong on this, but I believe the form of Taylors theorem that you stated relies on $C^3$, though the claim still holds in the $C^2$ case, by using Taylors theorem with remainder and that positive definite matrices remain pd under small perturbations, likewise for negative definite. – Andrew Aug 02 '23 at 15:53
  • @AndrewZhang Nope, twice differentiability suffices (alas, it depends what you understand for a vector function to be "differentiable"); in fact, I only need twice differentiability a $x,$ rather than a neighbourhood of $x.$ See Cartan's "Differential Calculus" theorem 5.6.3 for Taylor and 8.2.1 for the condition that the second derivative is psd (since Cartan's defines maximum as $f(x) \geq f(x + h)$ rather than "$>$"), however the argument for pd is the same. – William M. Aug 02 '23 at 16:11
  • @AndrewZhang if you want special forms for the remainder, such as integral form or a mean-value type equality, you do require extra assumptions, $C^3$ being a common one. – William M. Aug 02 '23 at 16:14