We need to apply Ito's lemma to the function $g(X_t)$, where $g(x) = \int_0^x\frac{1}{\sigma(u)}\ du$, and where $dX_t = \sigma(X_t)dW_t$. Notice that there is no $t$-dependence in the function $g$, so the first term $f_t(X_t,t)dt$ of Ito's lemma vanishes, and the lemma reduces to
$$dg(X_t) = g_x(X_t)dX_t + \frac{1}{2}g_{xx}(X_t)\sigma(X_t)^2dt.$$
Consider the first term $g_x(X_t)dX_t$. Here $g_x$ is just the derivative of $g$, and by the fundamental theorem of calculus, we have
$$g_x(x) = \frac{d}{dx}g(x) = \frac{d}{dx}\int_0^x\frac{1}{\sigma(u)}\ du = \frac{1}{\sigma(x)},$$
and thus
$$g_x(X_t)dX_t = \frac{1}{\sigma(X_t)}dX_t = \frac{1}{\sigma(X_t)}\sigma(X_t)dW_t = dW_t.$$
Consider then the second term $\frac{1}{2}g_{xx}(X_t)\sigma(X_t)^2dt$. The expression $g_{xx}$ is nothing more but the second derivative of $g$, and we have by the chain rule
$$g_{xx}(x) = \frac{d}{dx}\frac{1}{\sigma(x)} = -\frac{1}{\sigma(x)^2}\sigma'(x).$$
We then obtain
$$\frac{1}{2}g_{xx}(X_t)\sigma(X_t)^2dt = \frac{1}{2}\left(-\frac{1}{\sigma(X_t)^2}\sigma'(X_t)\right)\sigma(X_t)^2dt = -\frac{1}{2}\sigma'(X_t)dt.$$
In total, we have
$$dg(X_t) = dW_t - \frac{1}{2}\sigma'(X_t)dt.$$