If $f$ and $g$ are both functions from the set $X$ to $X$ and $f\circ g$ is the identity function, does $g\circ f$ also have to be the identity map? How (if at all) does your answer change if $X$ is finite?
I know I probably have to do some reasoning involving injective/surjective functions but I am struggling to picture what's going on.
No, consider $f,g:[0,\infty)\to[0,\infty)$ defined by $g(x)=x+1$ and $f(x)=\max\{0,x-1\}$. We have $fg=\operatorname{id}$ and $g(f(x))=\max\{1,x\}$, which is not the identity function. (For instance $g(f(0))=1.$)
For finite sets, the answer is positive. In this case, $fg=\operatorname{id}$ implies that $g$ is injective and $f$ surjective. Since $X$ is finite, we may conclude that $f$ and $g$ are bijective and $g=f^{-1}$.
In general, if $f\circ g$ is an injective function, then $g$ is an injective function, and if $f\circ g$ is a surjective function, then $f$ is a surjective function. In our case, $f\circ g=Id$ gives that $g$ is an injective function and $f$ is a surjective function. If $X$ is a finite set, injectivity implies surjectivity, hence $g$ is a bijective map and $f\circ g(x)=x$ implies $f(x)=g^{-1}(x)$, from which $g\circ f=Id$ follows.
However, if $X$ is not a finite set we have counter-examples. For instance, we may take an injective map $g:[0,1]\to[0,1)$ and a surjective map $f$ for which $f(1)=1$ and $f([0,1))=[0,1]$ - just to be clear, $g$ removes a point from an interval and $f$ add it back - such that $f\circ g$ is the identity over $X=[0,1]$. In such a case, however, $1$ does not belong to $g\circ f(X)$, hence $g\circ f$ cannot be the identity over $X$.
if $X$ is a finite set, the claim is true because an injective function $X\rightarrow X$ is also surjective, hence bijective in this case.
This is not true in general, depending on what X is. If X is a vector space, for example, it can be shown that this is true (or at least, for finite dimensional vector spaces).
