By definition of cross product, if $\theta$ is the angle between vectors $\underline{a}$ and $\underline{b}$, then $$\sin\theta=\frac{|\underline{a}\times\underline{b}|}{|\underline{a}||\underline{b}|}.$$
For this cube we will take the origin to be one vertex, and the diagonal to be the line with equation $$\underline{r}=\lambda\left(\begin{matrix}1\\1\\1\end{matrix}\right).$$
Therefore the distance from a typical point $P(x,y,z)$ in the cube to this diagonal is $$|OP|\sin\theta=\frac{\left|\left(\begin{matrix}x\\y\\z\end{matrix}\right)\times\left(\begin{matrix}1\\1\\1\end{matrix}\right)\right|}{\sqrt{3}}=\frac{1}{\sqrt{3}}\left|\left(\begin{matrix}y-z\\z-x\\x-y\end{matrix}\right)\right|$$
The density of the cube is $\rho=\frac{m}{a^3}$.
Therefore the moment of inertia of the cube about the diagonal is $$\frac{m}{3a^3}\int_0^a\int_0^a\int_0^a\left((y-z)^2+(z-x)^2+(x-y)^2\right)dxdydz$$
$$=...$$
$$=\frac 16 ma^2$$