You might want to read about Lyapunov functions.
But for the question, it is simply that if $x$ were periodic with period $T$, then $x(T)=x(0)$, $x(T+s)=x(s)$ and thus also $x'(T)=x'(0)$, so that of course also because of equal inputs into $E$
$$
E(T)=H(x'(T),x(T))=H(x(0),x'(0))=E(0).
$$
This would be true for any function $H$ of $x$ and $x'$, but with this $H(x,v)=v^2+x^2$ you know from $\dot E=∇H(x,v)·(\dot x,\dot v)^T$ and by the fundamental theorem of calculus that
$$
E(T)-E(0)=\int_0^T\dot E(t)dt=\int_0^T2x'(t)\,(x''(t)+x(t))dt=-2\int_0^Tx'(t)^4dt\le 0
$$
This can only be $0$ if $x'$ is the zero function, $x$ a constant which then also would have to be $0$.
Approximate Amplitude Evolution
For small amplitudes the equation is almost a harmonic oscillator. Approximate this as $x(t)=r(t)\cos(t)$ with a slowly changing $r$. Then $E(t)\approx r(t)^2$ and
$$
r(t+\pi)^2-r(t)^2\approx-2r(t)^4\int_0^\pi\sin^4(s)\,ds
=-\frac{3\pi}4r(t)^4
$$
as $$8\sin^4(s)=2(1-\cos2s)^2=2-4\cos2s+(1+\cos4s).$$
The resulting recursion $$u_+=u-cu^2\approx u(1-cu_+),$$ $u=r(t)^2$, $u_+=r(t+\pi)^2$, $c=\frac{3\pi}4$, can in the indicated approximation be solved via
$$\frac1{u_+}=\frac1u+c,$$ which extends iteratively to
$$
r(t+k\pi)^{-2}=r(t)^{-2}+k\pi\frac{3}4
$$
By linear interpolation
$$
r(t)=\frac{r(0)}{\sqrt{1+\frac34 r(0)^2·t}}
$$
This gives the approximate solution
$$
x(t)=\frac{r(0)\cos t}{\sqrt{1+\frac{3}4r(0)^2·t}}
$$
Already this very basic approximation gives visually convincing results, for instance in this visualization for $x(0)=a=0.5$, $x'(0)=0$:
import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
f = lambda y,t: [ y[1], -y[0] - y[1]**3 ]
t = np.arange(0,60,0.05)
a=0.5
sol = odeint(f, [ a, 0], t)
plt.plot(t, sol[:,0], 'b', label='exact')
plt.plot(t, a*np.cos(t)/np.sqrt(1+0.25*a*a*3*t), 'g', label='1.ord, 1. freq')
plt.xlabel('t'); plt.ylabel('x'); plt.grid(); plt.legend(loc='best'); plt.show()
