If we take $x$ outer, first we find the smallest and biggest $x$ in the region. The smallest $x$ happens at the left corner, $(1,1)$, and the biggest happens at the right corner, $(2,1)$. To find the limits on $y$ we draw a curve of constant $x$, a vertical line, through the region. Near the left corner the line would intersect the boundary of the region at the hyperbola $y=1/x$ and the line $y=x$, but as the vertical line moves to the right things change for $x>\sqrt2$. After this point the line intersects the boundary of the region at $y=x/2$ and $y=2/x$, so for $\sqrt2<x<2$ the limits on $y$ are different so the $x$ integral has to be broken up into $2$ pieces:
$$\int\int_D\frac{x^2+2x}{\sqrt y}d^2A=\int_1^{\sqrt2}\int_{1/x}^x\frac{x^2+2x}{\sqrt y}dy\,dx+\int_{\sqrt2}^2\int_{x/2}^{2/x}\frac{x^2+2x}{\sqrt y}dy\,dx$$
The observant reader will note that we were fortunate that the upper and lower corners both happened when $x=\sqrt2$; if the boundary $y=2/x$ were changed to $y=3/x$ for example, we would have had to integrate $3$ pieces.
If we instead took $y$ outer, the smallest $y$ happens at the lower corner, $(\sqrt2,1/\sqrt2)$ and the biggest $y$ at the upper corner, $(\sqrt2,\sqrt2)$. To find the limits on $x$ for a given $y$ we draw a curve of constant $y$, a horizontal line, through the figure. For $1/\sqrt2<y<1$ the line intersects the boundary of the region at $x=1/y$ and $x=2y$, but for $1<y<\sqrt2$, the line intersects the boundary of the region at $y=x$ and $y=2/x$. Again we were fortunate that the left and right corners happened at the same $y$. So this way we get
$$\int\int_D\frac{x^2+2x}{\sqrt y}d^2A=\int_{1/\sqrt2}^1\int_{1/y}^{2y}\frac{x^2+2x}{\sqrt y}dx\,dy+\int_1^{\sqrt2}\int_y^{2/y}\frac{x^2+2x}{\sqrt y}dx\,dy$$
The integral can be done in one piece if we are allowed to use the Jacobian. Let $u=xy$, $v=x/y$. Then $x=(uv)^{1/2}$, $y=(u/v)^{1/2}$ and
$$\begin{align}d^2A&=dx\,dy=\left|\det\begin{bmatrix}\frac{\partial x}{\partial u}&\frac{\partial x}{\partial v}\\\frac{\partial y}{\partial u}&\frac{\partial y}{\partial v}\end{bmatrix}\right|du\,dv\\
&=\left|\det\begin{bmatrix}\frac12\left(\frac vu\right)^{1/2}&\frac1{2(uv)^{1/2}}\\\frac12\left(\frac uv\right)^{1/2}&-\frac12\left(\frac u{v^3}\right)^{1/2}\end{bmatrix}\right|du\,dv=\frac1{2v}du\,dv\end{align}$$
Now the region is bounded by the hyperbolas $xy=u=1$ and $xy=u=2$ and the lines $x/y=v=1$ and $x/y=v=2$, so it's rectangular in the $(u,v)$ coordinate system and we get
$$\int\int_D\frac{x^2+2x}{\sqrt y}d^2A=\int_1^2\int_1^2\frac{uv+2(uv)^{1/2}}{\left(\frac uv\right)^{1/2}}\frac1{2v}dv\,du$$
All $3$ ways you should get
$$\int\int_D\frac{x^2+2x}{\sqrt y}d^2A=-\frac{32}{21}2^{3/4}-\frac{272}{105}2^{1/4}+\frac{772}{105}$$