I have a problem in evaluating the double integral $$\int\limits_{-\infty}^{\infty}\int\limits_{-\infty}^{\infty}e^{-5x^2-5y^2+8xy}dxdy$$
I tried to use polar coordinates
$e^{-(r^2(5-8\sin(\theta)\cos(\theta))}$ but still unsolvable
Any help please
I have a problem in evaluating the double integral $$\int\limits_{-\infty}^{\infty}\int\limits_{-\infty}^{\infty}e^{-5x^2-5y^2+8xy}dxdy$$
I tried to use polar coordinates
$e^{-(r^2(5-8\sin(\theta)\cos(\theta))}$ but still unsolvable
Any help please
Assume that $Q(x,y)=Ax^2+Bxy+Cy^2$ is a positive-definite quadratic form ($B^2<4AC$ and $A,C>0$) associated to the matrix $M=\left(\begin{smallmatrix} A & B/2 \\ B/2 & C\end{smallmatrix}\right)$. By the spectral theorem $M$ is conjugated to a diagonal matrix $\left(\begin{smallmatrix} \lambda_1 & 0 \\ 0 & \lambda_2\end{smallmatrix}\right)$ via an isometry, such that
$$ \iint_{\mathbb{R}^2}\exp(-Q(x,y))\,dx\,dy = \iint_{\mathbb{R}^2}\exp\left(-\lambda_1 x^2-\lambda_2 y^2\right)\,dx\,dy $$ equals, via Fubini's theorem, $\frac{\pi}{\sqrt{\lambda_1\lambda_2}}$. On the other hand $\lambda_1\lambda_2 = \det(M) = AC-\frac{B^2}{4}$, so $$ \iint_{\mathbb{R}^2}\exp(-Q(x,y))\,dx\,dy = \frac{2\pi}{\sqrt{4AC-B^2}}. $$ In your case $B=-8$ and $A=C=5$, so $$ \iint_{\mathbb{R}^2} e^{-5x^2-5y^2+8xy}\,dx\,dy = \frac{2\pi}{\sqrt{100-64}} = \color{red}{\frac{\pi}{3}}.$$
Note that $$I=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-5x^2-5y^2+8xy}dxdy =\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} e^{-\frac12(x+y)^2-\frac92(x-y)^2}dxdy $$
and let $u=\frac1{\sqrt2}(x+y)$, $v=\frac3{\sqrt2}(x-y)$ to obtain
$$I = \frac13\int_{-\infty}^{\infty}e^{-u^2}du\int_{-\infty}^{\infty} e^{-v^2}dv = \frac{(\sqrt\pi )^2}3 = \frac\pi3 $$
where $\int_{-\infty}^{\infty}e^{-u^2}du={\sqrt\pi}$ is used.
Since the OP tried evaluating the double integral by transforming to polar coordinates, I thought that it would be instructive to show that the we can indeed proceed by the proposed polar coordinate transformation.
Note that we have
$$\begin{align} \int_{-\infty}^\infty \int_{-\infty}^\infty e^{-5x^2-5y^2+8xy}\,dx\,dy&=\int_0^{2\pi}\int_0^\infty e^{-(5-8\cos(\phi)\sin(\phi))\rho^2}\,\rho\,d\rho\,d\phi\\\\ &=\frac12\int_0^{2\pi}\frac1{5-4\sin(2\phi)}\,d\phi\\\\ &=\frac12\int_0^{2\pi}\frac1{5-4\sin(\phi)}\,d\phi\tag1\\\\ &=\frac\pi3\tag2 \end{align}$$
where in going from $(1)$ to $(2)$ one can use either the tangent half-angle substitution or contour integration in the complex plane.