I took $1+x^3$ as $t^2$ . I also split $x^5$ as $x^2 .x^3$ . Then I subsituted the differentiated value in in $x^2$ . I put $x^3$ as $1- t^2$ . I am getting the last step as $2/9[\sqrt{1+x^3}x^3 ]$ but this is the wrong answer , i should get $2/9[\sqrt{1+x^3}(x^3 +2)]$.
Please help me. Thanks
It looks like you've only made a minor mistake, but since I couldn't tell exactly where it happened, I have provided a step-by-step solution starting with the substitution you chose.
If $t^2=1+x^3$, then $2t\,dt=3x^2\,dx$, so \begin{align*} \int \frac{x^5\,dx}{\sqrt{1+x^3}}&=\int \frac{x^3\cdot 3x^2\,dx}{3\sqrt{1+x^3}}\\ &=\int \frac{2t(t^2-1)\,dt}{3t}\\ &=\frac{2}{3}\int (t^2-1)\,dt\\ &=\frac{2}{3}\left(\frac{t^3}{3}-t\right)+C\\ &=\frac{2}{9}\left(t^3-3t\right)+C\\ &=\frac{2}{9}\left((1+x^3)\sqrt{1+x^3}-3\sqrt{1+x^3}\right)+C\\ &=\frac{2}{9}(x^3-2)\sqrt{1+x^3}+C. \end{align*}
This is an integration by parts problem, but first, set $u=x^3$. Then $ du = 3x^2 dx $, so the integral becomes $$ \frac{1}{3}\int \frac{u}{\sqrt{1+u}} \, du, $$ which you can see how to do parts on: $$ \int \frac{u}{\sqrt{1+u}} \, du = 2u\sqrt{1+u} - 2\int \sqrt{1+u} \, du = 2u\sqrt{1+u} - \frac{4}{3}(1+u)^{3/2} +C, $$ which you can also write as $\frac{2}{3}(u-2)\sqrt{1+u} + C$. Then the final answer is $$ \frac{2}{9}(x^3-2)\sqrt{1+x^3} + C. $$
Substitute $t=1+x^3$ and $dt=3x^2$:
\begin{align*} \int \frac{x^5 dx}{\sqrt{1+x^3}}&=\frac{1}{3}\int \frac{(t-1) dt}{\sqrt{t}}\\ &=\frac{1}{3}\int \left( \sqrt{t}-\frac{1}{\sqrt{t}}\right)dt\\ &=\frac{1}{3}\left( \frac{2}{3}t^\frac{3}{2}-2\sqrt{t}\right)\\ &=\frac{2}{9}(x^3-2)\sqrt{1+x^2} \end{align*}
And of course don't forget the constant of integration.
