1

The determinant of the Hessian is $0$, so we can't use this method but that's the only method I can see from the textbook. How else can we show that this is a saddle point?

1 Answers1

1

Along the line $x=0$, the function is concave up, looking like $y^6$. Along the line $y=0$, the function is $x^6-x^4=x^4(x^2-1)$, which near 0 looks like $-x^4$, which is concave down.

vadim123
  • 82,796