That is not true in general. Choose $g(x)$ some function that is injective but not surjective. Choose some $f(x)$ such that $f(g(x))=x$; this is possible since $g$ is injective.
So the image of $g$ is not the entire domain, thus it is impossible that $g(f(x))=x$ for all $x$ in the domain.
However, your claim does hold if the domain is finite; This is because $f(g(x))=x$ implies $f$ surjective, $g$ injective. But over a finite domain this implies that $f$,$g$ are bijections, and from $f\circ{}g=id$ we have
$g\circ{}f\circ{}g=g\circ{}id=g$
taking inverse from the right, we get $g\circ{}f=id$.
A concrete example:
choose $g:\mathbb{R}\to\mathbb{R}$ as $g(x)=e^x$.
choose $f:\mathbb{R}\to\mathbb{R}$ as $f(x)=\ln(x)$ for all $x>0$, and $f(x)=42$ for all $x\le{}0$.
Then $g(x)>0$ for all $x\in\mathbb{R}$. So $f(g(x))=\ln(e^x)=x$ for all $x\in\mathbb{R}$.
On the other hand, $g(f(-1))=g(42)=e^{42}\ne{}-1$