Do Carmo's formulation is ambiguous, and he does not mean that the Gauss Lemma implies that the boundary of a geodesic ball is an hypersurface. The author might have prefered this formulation:
The boundary of a geodesic ball, contained in a normal neighbourhood, is an hypersurface, and by Gauss Lemma, it is normal to geodesic rays starting from $p$.
The fact that small geodesic balls are hypersurfaces comes from the very definition of a normal neighbourhood. Indeed, for $V\subset T_pM$, being a normal neighbourhood of $0 \in T_pM$ means that $\exp_p \colon V \to \exp_p(V) \subset M$ is a diffeomorphism. Thus, as a sphere of radius $\varepsilon$ in $T_pM$ is an hypersurface and for $\varepsilon$ small enough, it is contained in $V$, this imply that its image is a compact hypersurface of $\exp_p(V)$, which is open in $M$. It follows that it is an hypersurface of $M$.
The fact that it is normal to geodesic rays is a direct consequence of Gauss Lemma. If $v \in T_pM$ is a non-zero vector and if $w \perp v$, then, by Gauss Lemma:
$$
\forall t, \langle \mathrm{d}\exp_p(tv)v,\mathrm{d}\exp_p(tv)w\rangle = \langle v,w\rangle = 0
$$
and therefore, $\mathrm{d}\exp_p(tv)w$ is orthogonal to $\mathrm{d}\exp_p(tv)v$, which is the tangent vector of $\gamma(t) = \exp_p(tv)$.
Comment the other answer claims a wrong statement. It is false that the exponential map is an immersion. For example, on the unit sphere, if $v$ and $w$ are orthonormal vector at $p \in S^n$, then $\mathrm{d}\exp_p(\pi v) w = 0$ by easy Jacobi fields analysis: indeed $\mathrm{d}\exp_p(tv)(tw) = \sin t W$ where $W$ is the parallel transport of $w$ along $\exp(tv)$. Therefore, $\mathrm{d}\exp_p(\pi v)$ is not injective. However, it is true that in non-positive curvature, it is.