I think your main problem is with the notion:
$$ \lim\limits_{x \to \infty} \tan^{-1} (\frac{x}{4}) \neq \lim\limits_{x \to \infty} \frac{1}{\tan (\frac{x}{4})}$$
instead if $f(x) = \tan (x)$
$$\tan^{-1} (x) = f^{-1}(x)$$
For clarity I will use $\tan^{-1} (x) = \arctan (x)$
Knowing that let's take the limit:
First lets substitute $t = \frac{x}{4}$ (as suggested before):
$$ \lim\limits_{x \to \infty} \arctan (\frac{x}{4}) = \lim\limits_{t \to \infty} \arctan (t) , t = \frac{x}{4}$$
notice that $$ \lim\limits_{t \to \infty} t = \lim\limits_{x \to \infty} \frac{x}{4} = \infty $$
now we notice that since the $\arctan (x) $ function should produce a number that if input into the tangent function will output $x$, and since the tangent function has a range from $(-\infty , \infty)$ between $ (\frac{-\pi}{2},\frac{\pi}{2}) $, the inverse tangent if given an input from $(-\infty , \infty)$ you will get a value from $ (\frac{-\pi}{2},\frac{\pi}{2}) $. As we take $ t \to \infty$, we get closer and closer to $\frac{\pi}{2}$, therefore:
$$ \lim\limits_{x \to \infty} \arctan (\frac{x}{4}) = \lim\limits_{t \to \infty} \arctan(t) = \frac{\pi}{2} $$