Consider $h>0$ then there is $n$ so that $ \frac{1}{(n+1)\pi +\pi/2} < h\le \frac{1}{n\pi + \pi/2}$. In this interval (called $I_n$), $\cos(1/t)$ is positive (resp. negative) if $n$ is odd (resp. even). Also
$$\int_0^h \cos\left(\frac 1t\right) dt = \sum_{k=n+1}^\infty \int_{I_k} \cos\left(\frac 1t\right) dt + \int_{\frac{1}{(n+1)\pi + \pi/2}}^h \cos\left(\frac 1t\right) dt$$
Note that the last term is bounded by $\frac{2}{\pi n^2}$. On the other hand, if we let
$$a_k = \int_{I_k} \cos\left(\frac 1t\right) dt,$$
and $a_k$ is an alternating sequence and $a_k \to 0$. If $l> k$, then $|a_k| > |a_l|$ (see below). Thus we have
$$\left|\sum_{k=n+1}^\infty a_k\right| \le |a_{n+1}| \Rightarrow \left| \sum_{k=n+1}^\infty \int_{I_k} \cos\left(\frac 1t\right) dt\right| \le \int_{I_{n+1} }\left| \cos\left(\frac 1t\right) \right| dt \le \frac{2}{\pi n^2}. $$
As $h \in I_n$, $h > \frac{1}{(n+1)\pi + \pi/2}> \frac{1}{2\pi n} \Rightarrow \frac{1}{h} < 2\pi n$. Thus
$$\left|\frac 1h\int_0^h \cos \left(\frac 1t \right) dt \right| \le (2\pi n) \frac{4}{\pi n^2} = \frac{8}{n}. $$
As $h\to 0$, $n\to \infty$ and so the limit goes to zero.
Remark: If we do the substitution $u = 1/t$, then
$$a_k = \int_{I_k} \cos\left(\frac 1t\right) dt = \int_{k\pi +\pi/2}^{(k+1)\pi + \pi/2} \frac{\cos u}{u^2} \mathrm du \Rightarrow |a_k| = \int_{k\pi +\pi/2}^{(k+1)\pi + \pi/2} \frac{|\cos u|}{u^2} \mathrm du.$$
Now $|\cos u|$ is $\pi$-periodic, $|a_k|$ is strictly decreasing.