Find this integral $$I(a,b)=\iiint_{x^2+y^2+z^2\le 1}(ax+by)^2\,dx\,dy\,dz$$
since $$(ax+by)^2=a^2x^2+b^2y^2+2abxy$$ so $$I=I_{1}+I_{2}+I_{3}$$ where $$I_{1}=\iiint_{x^2+y^2+z^2\le 1}ax^2\,dV=a\iiint_{x^2+y^2+z^2\le 1}x^2\,dV$$
since $$\iiint_{x^2+y^2+z^2\le 1}x^2\,dV=\iiint_{x^2+y^2+z^2\le 1}y^2\,dV$$
maybe have other methods? Thank you
With (almost) no computations:
The domain $x^2+y^2+z^2\leqslant1$ is invariant by the rotations of the plane $(x,y)$ hence $$I(a,b)=(a^2+b^2)\,J,$$ where $$ J=\int\!\!\!\iint_{x^2+y^2+z^2\leqslant1}x^2\mathrm dx\mathrm dy\mathrm dz. $$ By symmetry with respect to the $(x,y,z)$ axes, $3J=K(1)$, where $$ K(t)=\int\!\!\!\iint_{x^2+y^2+z^2\leqslant t^2}(x^2+y^2+z^2)\mathrm dx\mathrm dy\mathrm dz. $$ By differentiation, using the fact that the surface of the sphere of radius $t$ is $4\pi\,t^2$, $$ K'(t)=4\pi\,t^2\cdot t^2=4\pi\,t^4. $$ Since $K(0)=0$, for every nonnegative $t$, $$ K(t)=\frac45\pi\,t^5, $$ and, finally, $$ I(a,b)=\frac13\,\frac45\pi\,(a^2+b^2). $$
Expanding out, your integral is $$a^2 \iiint_{x^2+y^2+z^2\le 1}x^2\,dV - 2ab \iiint_{x^2+y^2+z^2\le 1}xy\,dV + b^2 \iiint_{x^2+y^2+z^2\le 1}y^2\,dV$$ The middle term is zero since for fixed $y$ and $z$ the integrand is odd in $x$ and the domain of integration is a symmetric interval in $x$. Furthermore, by symmetry the first and third integrals are the same, so your answer will be $$ (a^2 + b^2) \iiint_{x^2+y^2+z^2\le 1}x^2\,dV $$ The integrand is constant in $y$ and $z$, so if we integrate in those variables first, we introduce a factor given by the area of the $yz$-cross section for fixed $x$, namely $\pi(1 - x^2)$ (This is because the cross section is the circle $y^2 + z^2 = 1 - x^2$, of radius $\sqrt{1 - x^2}$). So the answer is $$(a^2 + b^2) \int_{-1}^1 \pi(1 - x^2)x^2\,dx$$ $$= \pi(a^2 + b^2)\int_{-1}^1 (x^2 - x^4)\,dx$$ $$= \pi(a^2 + b^2) \bigg({x^3 \over 3} - {x^5 \over 5}\bigg)\bigg|_{x = -1}^{x=1}$$ $$= {4 \over 15}\pi(a^2 + b^2)$$
