Maple shows that $$ \int_0^1 \int_0^{1-x} \arctan\left(\sqrt{y/x}\right)/\sqrt{xy} \, \mathrm d y \,\mathrm d x = \pi^2/4. $$ It looks simple but seems rather tedious to do compute manually. Is there easy proof for this?
BTW, converting to polar system does not seem to help. It gives $$ \int_0^{\pi/2} \left(\theta \left/\left(\sqrt{\cos(\theta)\sin(\theta)}(\sin(\theta)+\cos(\theta)\right) \right. \right) \mathrm d \theta, $$ which both WolframAlpha and Maple cannot solve.
Hint:
Let $x=r^2\cos^2t$ and $y=r^2\sin^2t$ then ${\bf J}=2r^3\sin2t$ and $$\int_0^1 \int_0^{1-x} \arctan\left(\sqrt{y/x}\right)/\sqrt{xy} \, \mathrm d y \,\mathrm d x = \int_0^{\frac{\pi}{2}}\int_0^14tr\,dr\,dt=\color{blue}{\dfrac{\pi^2}{4}}$$
Your domain of integration is $D=\{(x,y); 0\leq x\leq 1; 0\leq y\leq 1-x\}$. Using Fubini, your integral is hence $$I=\int_0^1 (\int_0^{1-y}\arctan(\sqrt{y/x})\frac{dx}{\sqrt{xy}})dy$$
In the last integral, change the name of the variables: replace $x$ by $y$ and $y$ by $x$. You get $$I=\int_0^1 \int_0^{1-x}\arctan(\sqrt{x/y})\frac{dy}{\sqrt{xy}})dx$$ Hence
$$I=\int_0^1 \int_0^{1-x}\arctan(\sqrt{y/x})\frac{dy}{\sqrt{xy}})dx= \int_0^1 \int_0^{1-x}\arctan(\sqrt{x/y})\frac{dy}{\sqrt{xy}})dx$$
Now remember you have $$\arctan(u)+\arctan(1/u)=\frac{\pi}{2}$$ for $u>0$, and compute $2I$.
The "manual" way isn't so bad, but definitely not as elegant as the other answers.
$$\begin{align*} I &= \int_0^1 \int_0^{1-x} \frac{\arctan \sqrt{\frac yx}}{\sqrt{xy}} \, dy \, dx \\ &= 2 \int_0^1 \int_0^{\sqrt{\tfrac{1-x}x}} \arctan u \, du \, dx & u=\sqrt{\frac yx} \\ &= \int_0^1 \left(2 \sqrt{\frac{1-x}x} \arctan \sqrt{\frac{1-x}x} + \log x\right) \, dx & \rm IBP \\ &= \int_0^\infty \left(\frac{4v^2\arctan v}{\left(1+v^2\right)^2} \, dv - \frac{2v\log\left(1+v^2\right)}{\left(1+v^2\right)^2}\right) \, dv & v = \sqrt{\frac{1-x}x} \\ &= 4 \int_0^\tfrac\pi2 s\sin^2s \, ds - 2 \int_0^\infty \frac{v}{(1+v^2)^2} \, dv & s=\arctan v; \rm IBP \\ &= 4 \left(\frac{\pi^2}{16}+\frac14\right) - 2 \left(\frac12\right) = \boxed{\frac{\pi^2}4} \end{align*}$$
