$$\int_{0}^{1} t^2 \sqrt{(1+4t^2)}dt$$
my attempt
$$t = \frac{1}{2}\tan(u)$$
$$dt = \frac{1}{2}\sec^2(u)du\\$$
$$\begin{align} \int_{0}^{1} t^2 \sqrt{(1+4t^2)}dt&=\int_{0}^{1} \frac{\tan^2(u)}{4} \sqrt{1+\tan^2(u)}\frac{1}{2}\sec^2(u)du\\ &=\frac{1}{8}\int_{0}^{1} \tan^2(u)\sec^{3}(u)du\\ &= \frac{1}{8} \int_{0}^{1} (\sec^2(u) - 1)(\sec^{3}(u))du\\ &=\frac{1}{8}\int_{0}^{1} \sec^5(u)du - \frac{1}{8} \int_{0}^{1}\sec^3(u)du \end{align}$$
what now?
To compute the integral in question first consider that $$\cosh^2(x)-\sinh^2(x)=1,$$ so $$\cosh^2(x)=1+\sinh^2(x).$$
Making the substitution $t=\frac{\sinh(x)}{2},$ then $4t^2=\sinh^2(x),$ and we get $dt=\frac{\cosh(x)}{2}dx$
$$\int t^2 \sqrt{(1+4t^2)}dt=\frac{1}{8}\int{\sinh^2(x)}\sqrt{1+\sinh^2(x)}\cosh(x)dx$$
$$=\frac{1}{8}\int{\sinh^2(x)}\cosh^2(x)dx=\frac{1}{8}\left(\frac{1}{32}\sinh(4x)-4x\right) +C.$$
I leave the change of parameter for $t=0$ and $t=1$ for you to finish.
We substitute $$t=\frac{\tan(u)}{2}$$ then$$dt=\frac{\sec^2(u)}{2}du$$ and $$\sqrt{4t^2+1}=\sqrt{\tan^2(u)+1}=\sec(u)$$ and our integral will be $$\frac{1}{2}\int\frac{1}{4}\tan^2(u)\sec^3(u)du$$ and this is $$\frac{1}{8}\int\sec^3(u)(\sec^2(u)-1)du$$ and then we need the formula $$\int\sec^m(u)du=\frac{\sin(u)\sec^{m-1}(u)}{m-1}+\frac{m-2}{m-1}\int\sec^{m-2}du$$
Hint. I would say
$\int \sec^5u\,du = \int\frac{\cos u\,du}{\cos^6u}=\int\frac{\cos u\,du}{(1-\sin^2u)^3}$ and after substitution
$\sin u = v,\quad \cos u \,du = dv$
it will be an integral part of the rational function.
Edit:
I will add: The given integral is the so-called binomial integral with a specific solution. See
$$\begin{align} \int t^2 \sqrt{(1+4t^2)}dt&=\frac{1}{8}\int{\sinh^2(x)}\sqrt{1+\sinh^2(x)}\cosh(x)dx\\ &=\frac{1}{8}\int{\sinh^2(x)}\cosh^2(x)dx=\frac{1}{8\times 32}\left(\sinh(4x)-4x\right) +C \\ &= \frac{1}{256}\left((4\sinh(x)\cosh^3(x) + 4\sinh^3(x)\cosh(x)) - 4x\right) + C\\ &=\left[\frac{1}{256}\left((4(2t)(\sqrt{1+4t^2})^3 + 4(2t)^3\sqrt{1+4t^2}) - 4\sinh^{-1}(2t)\right)\right]_{0}^{1} \approx 0.6063 \end{align}$$
