It depends on the definition of order of an element.
My preferred definition is
Let $G$ be a group and $g\in G$. The order of $g$ is the number of elements of $\langle g\rangle$, if finite; otherwise we say the order is infinite.
Given $g\in G$, we can define $\varphi_g\colon\mathbb{Z}\to G$ by
$$
\varphi_g(n)=g^n \qquad(n\in\mathbb{Z})
$$
and $\varphi_g$ is easily seen to be a homomorphism; the image of $\varphi_g$ is exactly $\langle g\rangle$.
The homomorphism theorem then says that $\varphi_g$ induces an isomorphism between $\mathbb{Z}/\ker\varphi_g$ and $\langle g\rangle$. A known fact about subgroups of $\mathbb{Z}$ gives that $\ker\varphi_g=k\mathbb{Z}$, for a unique $k\ge0$.
If $k=0$, $\varphi_g$ is injective and so $g$ has infinite order.
Otherwise $k$ is exactly the order of $g$. In this case, $g^k=\varphi_g(k)=1$ and, for $0<n<k$, $g^n\ne1$, because $n\notin k\mathbb{Z}=\ker\varphi_g$. Another consequence, in this case, is that if $g^n=1$, then $k\mid n$.
Of course, if $G$ is finite, also $\langle g\rangle$ is finite and so the order of $g$ divides $|G|$ by Lagrange's theorem.
Defining the order of $g$ as the least positive integer $k$, if it exists, such that $g^k=1$, forces proving again, but in a complicated way, that the subgroups of $\mathbb{Z}$ are of the form $n\mathbb{Z}$. I see no reason for not using the homomorphism theorem, which is one of the most basic tools in group theory.
However, as already said in other answers or comments, when $G$ is finite there are two distinct positive integers $m<n$ such that $g^m=g^n$, so $g^{n-m}=1$. Therefore the order of $g$ is finite, according to the other definition.