I am trying to use a second derivative test and I am stuck on how to proceed.
Below is how far I have gotten on the question:


This is my solution using an online graphing tool: I am still unsure how to do this mathematically.

I am trying to use a second derivative test and I am stuck on how to proceed.
Below is how far I have gotten on the question:


This is my solution using an online graphing tool: I am still unsure how to do this mathematically.

At critical points, we have the following system:
$$f_x=0$$
$$f_y=0$$
Assuming you want to know about $f(x,y) = (xy)e^{-2x-4y}$ rather than $f(x,y) = (xy)e^{-x^2-y^4}$, this means
$$(y-2xy)e^{-2x-4y} = 0$$
$$(x-4xy)e^{-2x-4y} = 0$$
which means
$$y(1-2x) = 0$$ (1)
$$x(1-4y) = 0$$ (2)
since $e^{-2x-4y}$ is never $0$.
From (1),
$y = 0$ or $x = \frac{1}{2}$
Substituting $y = 0$ into (2) gives
$$x = 0$$ so we have a critical point at $(0,0)$
Substituting $x = \frac{1}{2}$ into (2) gives
$$y = \frac{1}{4}$$
so the other critical point is $(\frac{1}{2}, \frac{1}{4})$
Now just compute $$D(a,b) = f_{xx}(a,b)f_{yy}(a,b) - {[f_{xy}(a,b)]}^2$$ for each point to classify it:
If $D(a,b) \gt 0$ and $f_{xx}(a,b) \gt 0$ there's a relative minimum at $(a,b)$
If $D(a,b) \gt 0$ and $f_{xx}(a,b) \lt 0$ there's a relative maximum at $(a,b)$
If $D(a,b) \lt 0$ there's a saddle point at $(a,b)$
If $D(a,b) = 0$ then the point $(a,b)$ may be a relative minimum, relative maximum or a saddle point. Other techiniques would be needed to find out which.
Hints:
Assuming
$$f(x,y)=xye^{-x^2-y^4}\;:$$
$$\begin{align*}f'_x&=ye^{-x^2-y^4}\left(1-2x^2\right)=0\iff\begin{cases}y=0&,\;\text{or}\\{}\\x=\pm\frac1{\sqrt2}\end{cases}\\{}\\f'_y&=xe^{-x^2-y^4}\left(1-4y^4\right)=0\iff\begin{cases}x=0&,\;\text{or}\\{}\\y=\pm\frac1{\sqrt2}\end{cases}\\{}\\ f_{xy}'&=(1-2x^2)e^{-x^2-y^4}\left(1-4y^4\right)\\{}\\ f_{x^2}'&=-2xye^{-x^2-y^4}\left(3+2x^2\right)\end{align*}$$
etc.
HINT: Take the first derivative using the product rule three times along with the chain rule. Repeat to obtain the second derivative. Set $y$ to 0. Solve for $x$.