Exercise
Investigate the equilibrium points of
$\dot{x} = y \left( 16 \left(2x^2+2y^2-1 \right) -1 \right)$
$\dot{y} = x - \left(2x^2+2y^2-1 \right)\left(16x-4\right)$
and classify them according to their linear approximations.
Show that the homoclinic paths through $(0,0)$ are given by
$\left(x^2 + y^2 -\frac{1}{2}x \right)^2 - \frac{1}{16}\left(x^2+y^2\right)=0$
and that one homoclinic path lies within the order. Draw the phase portrait.
(Hint: Find the Hamiltonian function of the system)
Attempt
I have calculated the equilibrium points to be $(0,0), (\frac{3+\sqrt{3}}{8},0),(\frac{3-\sqrt{3}}{8},0)$
For the classifications of the equilibrium points I calculated the linearized system around each point and found that
The point $(\frac{3+\sqrt{3}}{8},0)$ is unstable saddle point for the almost linear system (original system)
The point $(\frac{3-\sqrt{3}}{8},0)$ is stable center point for the almost linear system.
The point $(0,0)$ is stable center point for the linearized system. This means that the Hartman-Grobman theorem for system linearization cannot be applied. Q1: What can I do to determine the classification of this point for the almost linear system?
For the homoclinic path I calculated the Hamiltonian function of the original system to be
$H(x,y) = 8x^4 + 8y^4 -8x^3 + \frac{3}{2}x^2 - \frac{1}{2}y^2 +16x^2y^2 -8xy^2$
If I set $H(x,y)=0$ then I get the equation that is supposed to represent the homoclinic paths but I do not understand why that is true. Q2: Can someone explain?
Q3: I also don't understand what the phrase "one homoclinic path lies within the order" means.