We present first a direct approach here that applied Leibniz's Rule for Differentiating Under the Integral.
Let $I(a)$ be the function given by
$$I(a)=\int_a^\infty e^{-ax^2}\,dx \tag 1$$
Enforcing the substitution $x \to x/\sqrt{a}$ yields
$$\begin{align}
I(a)&=\frac{1}{\sqrt{a}}\int_{a^{3/2}}^\infty e^{-x^2}\,dx\\\\
&=\frac12 \sqrt{\frac{\pi}{a}}\left(1-\text{erf}(a^{3/2})\right) \tag 2
\end{align}$$
where the error function $\text{erf}(x)$ is given by
$$\text{erf}(x)=\frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2}\,dt$$
Differentiating the right-hand sides of $(1)$ and $(2)$ and equating results reveals
$$\begin{align}
-\int_a^\infty x^2e^{-ax^2}\,dx-e^{-a^3}&=-\frac14\sqrt{\frac{\pi}{a^3}}\left(1-\text{erf}(a^{3/2})\right)-\frac34 \sqrt{\pi}\text{erf}'(a^{3/2})\\\\
&=-\frac14\sqrt{\frac{\pi}{a^3}}\left(1-\text{erf}(a^{3/2})\right)-\frac32 e^{-a^3}\tag 3
\end{align}$$
Finally, solving $(3)$ for the term of interest, we obtain
$$\bbox[5px,border:2px solid #C0A000]{\int_a^\infty x^2e^{-ax^2}\,dx=\frac14\sqrt{\frac{\pi}{a^3}}\left(1-\text{erf}(a^{3/2})\right)+\frac12 e^{-a^3}}$$
as was to be shown.
We can derive the formula in question in a similar way. Note that we can write
$$\begin{align}
J(a)&=\int e^{-ax^2}\,dx\\\\
&=\frac12\sqrt{\frac{\pi}{a}}\text{erf}(\sqrt{a}\,x)+C(a)
\end{align}$$
where $C(a)$ is an integration constant. Now, upon differentiating with respect to $a$, we find
$$\begin{align}
-\int x^2 e^{-ax^2}\,dx&=-\frac14 \sqrt{\frac{\pi}{a^3}}\text{erf}(\sqrt{a}\,x)+\frac{x}{2a}e^{-ax^2}+C'(a) \tag 4\\\\
\end{align}$$
whereupon solving $(4)$ for the term of interest yields
$$\int x^2 e^{-ax^2}\,dx=\frac14 \sqrt{\frac{\pi}{a^3}}\text{erf}(\sqrt{a}\,x)-\frac{x}{2a}e^{-ax^2}-C'(a)$$
When evaluating the indefinite integral between lower and upper limits of integration, the contribution from $C'(a)$ is zero. Therefore, we can drop the integration constant and simply write
$$\bbox[5px,border:2px solid #C0A000]{\int x^2 e^{-ax^2}\,dx=\frac14 \sqrt{\frac{\pi}{a^3}}\text{erf}(\sqrt{a}\,x)-\frac{x}{2a}e^{-ax^2}}$$