$N(t)=$ number of arrivals in $[0,t)$. Then, if the inter arrival time between $k$ and $k-1$th arrivals is denoted by $S_k$, we have, $t=\sum_{k=1}^{N(t)}S_k$. For a Poisson process, $S_k$ are i.i.d~$\exp(\lambda)$ random variables. Then, by weak law of large numbers, $\lim_{t\to \infty}\frac{N(t)}{t}=\dfrac{1}{\lim_{N\to \infty}\dfrac{\sum_{k=1}^N S_k}{N}}=\frac{1}{\mathbb{E}S_1}=\lambda$ in probability. Actually by SLLN, it holds almost surely. Now, since $N(t)\sim Poi(\lambda t)$, $P(N(t)\ge 1)=1-e^{-\lambda t}\to 0$ as $t\to 0$. Thus, $\lim_{t\to 0}\frac{P(N(t)\ge 1)}{t}=\lambda,\ \lim_{t\to 0}\frac{P(N(t)\ge k)}{t}=0,\ k\ge 2$. Also, if we assume that $N(0)=0$, then, for every sample path $\omega$ of the Poisson process, we can find some $\epsilon>0$ such that $N(\epsilon;\omega)=0$ and then $\lim_{t\to 0}\frac{N(t;\omega)}{t}=0\implies \lim_{t\to 0}N(t)/t=0$ point wise if $N(0)=0$.
The quantity $$\lim_{t\to \infty}\frac{N(t)}{t}$$ is defined as the rate of a counting process.