Take a region of the 3D graph, and plot points on it. I would recommend the range $x \in \mathbb{R}:[-2,2]$ and $ y \in \mathbb{R}:[-2,2]$. This gives you some positive, and some negative. When you do this, you make a note of the symmetry of the graph, as well as the points of intersection with the xy-plane, since that gives you the bonds of integration.
Per request, here is the solution:
In the 1st quadrant of the XY-plane, the graph appears to be a triangle from $(0,0) \rightarrow (1,0) \rightarrow (0,1)$. Thus, the line of the constraint is $z=1-x-y$. Notice that the aboslute value signs are dropped, because we are dealing with the first quadrant only. This is due to the symmetry of the structure. In general, you would have to separate the integration into four pieces, and solve each separately.
$$\iiint\limits_R (xy+z^2)dV=\iint\limits_A\int_0^{1-x-y} (xy+z^2)dzdA$$
The z-constraint is performed first, which is why it is explicitly stipulated with its constraint equation.
$$\iint\limits_A \int_0^{1-x-y} (xy+z^2)dzdA=\iint\limits_A \left [ xyz+\frac{z^3}{3} \right ]_0^{1-x-y} dA$$
Algebra follows to finish substituting for $z$ and get the integrand solely interms of $x$ and $y$.
$$=\iint\limits_A \left [ xy(1-x-y)+\frac{(1-x-y)^3}{3}-0-0\right ]dA$$
Wolfram Alpha for $(1-x-y)^3$.
$$=\iint\limits_A \left [xy-x^2-xy^2+\frac{1}{3}*(-x^3-3x^2y+3x^2-3xy^2+6xy-3x-y^3+3y^2-3y+1)\right ]dA$$
Now we can apply the constraint to $y$. It is the line of intersection of the $z$ surface with the XY-plane: $y=1-x$
$$=\int \int_0^{1-x} \left [xy-x^2-xy^2+\frac{1}{3}*(-x^3-3x^2y+3x^2-3xy^2+6xy-3x-y^3+3y^2-3y+1)\right ]dydx$$
$$=\int \left [ \frac{x(1-x)^2}{2}-(1-x)x^2-\frac{x(1-x)^3}{3}+\frac{1}{3}*(-(1-x)x^3-\frac{3x^2(1-x)^2}{2}+3x^2(1-x)-x(1-x)^3+3x(1-x)^2-3x(1-x)-\frac{(1-x)^4}{4}+(1-x)^3-\frac{3(1-x)^2}{2}+(1-x)) \right ]dx$$
Now we can apply the true constraints on $x$, which are $\left [0,1\right]$.
$$=\int_0^1 \left [ \frac{x(1-x)^2}{2}-(1-x)x^2-\frac{x(1-x)^3}{3}+\frac{1}{3}*(-(1-x)x^3-\frac{3x^2(1-x)^2}{2}+3x^2(1-x)-x(1-x)^3+3x(1-x)^2-3x(1-x)-\frac{(1-x)^4}{4}+(1-x)^3-\frac{3(1-x)^2}{2}+(1-x)) \right ]dx$$
At this point, the algebra is just tedious. Just integrate after expanding all powers, and substitute.