I have to evaluate the following limit $$ \lim_{x \to 0}x \tan (xa+ \arctan \frac{b}{x})$$ I tried to divide tan in $\frac{sin}{cos}$ or with Hopital but I can't understand where I'm making mistakes. The final result is:
$\frac{b}{1-ab}$ if $ab \ne 1$
$- \infty$ if $ab=1$ and $a>0$
$+ \infty$ if $ab=1$ and $a<0$
If you perform the trigonometric expansion, you should end with $$y=x \tan \left(a x+\tan ^{-1}\left(\frac{b}{x}\right)\right)=x\frac{b \cos (a x)+x \sin (a x)}{x \cos (a x)-b \sin (a x)}$$ and now, there is a problem because of the denominator.
Using Taylor expansions, we have $$y=x \frac {b+ \left(a-\frac{a^2 b}{2}\right)x^2+\frac{1}{24} a^3 (a b-4)x^4+O\left(x^5\right) } { (1-a b)x+\frac{1}{6} \left(a^3 b-3 a^2\right)x^3+O\left(x^5\right) }$$ Using the long division $$y=\frac{b}{1-a b}+\frac{a (a b (a b-3)+3)}{3 (1-a b)^2}x^2+O\left(x^4\right)$$
So, if $ab=1$ we have a poblem.
Hint:
$$x\cdot\tan\left(ax+\arctan\dfrac bx\right)$$
$$=x\cdot \dfrac{\tan ax+\dfrac bx}{1-\tan ax\cdot\dfrac bx}$$ $$=\dfrac{x(x\tan ax+b)}{x-b\tan ax}$$
$$=\dfrac{x\tan ax+b}{1-b\cdot\dfrac{\tan ax}x}$$
Finally use $\lim_{y\to}\dfrac{\tan y}y=1$
Using that $\tan(x+y) = \frac{\tan(x) + \tan(y)}{1-\tan(x)\tan(y)}$, this limit is $$\lim_{x \to 0} x \frac{\tan(xa) + \tan\left(\arctan \left(\frac{b}{x}\right)\right)}{1-\tan(xa)\tan\left(\arctan \left(\frac{b}{x}\right)\right)}$$
Since $\tan(\arctan(x)) = x$, this simplifies to $$\lim_{x \to 0} x \frac{\tan(xa) + \frac{b}{x}}{1-\tan(xa)\frac{b}{x}} = \lim_{x \to 0} \frac{x^2\tan(xa) + bx}{x-\tan(xa)b}$$
From here, applying L'Hopital's rule, this becomes $$\lim_{x \to 0} \frac{2x\tan(ax)+ax^2\sec^2(ax) + b}{1-ab\sec^2(ax)}$$
Now if $ab \not= 1$, we can simply plug everything in to get $\frac{b}{1-ab}$, while if $ab = 1$, the denominator would approach $0$ from the negative side. If $a>0$, the numerator would be a positive constant, while if $a<0$, the numerator would be a negative constant. Therefore, when $ab=1$ and $a>0$, the limit goes to $-\infty$, while if $ab=1$ and $a<0$, the limit goes to $\infty$.
We have that by $\arctan x+\arctan \frac 1x= \pm \frac \pi 2$
$$x \tan \left(xa+ \arctan \frac{b}{x}\right)=x \tan \left(\pm\frac\pi 2+xa-\arctan \frac{x}{b}\right)=$$
$$=-\frac{x}{ \tan \left(xa-\arctan \frac{x}{b}\right)}=-\frac{x}{ xa-\arctan \frac{x}{b}} \frac{xa-\arctan \frac{x}{b}}{ \tan \left(xa-\arctan \frac{x}{b}\right)}$$
and since by standard limits
$$\frac{xa-\arctan \frac{x}{b}}{ \tan \left(xa-\arctan \frac{x}{b}\right)} \to 1$$
we only need to consider the first term that is for $ab\ne -1$
$$-\frac{x}{ xa-\arctan \frac{x}{b}}=-\frac{1}{ a-\frac1b\frac{\arctan \frac{x}{b}}{\frac x b}} \to\frac{b}{1-ab}$$
by standard limits and for $ab= 1$ by Taylor's expansion $\arctan (x)=x-\frac13x^3+o(x^3)$ $$-\frac{x}{ xa-\arctan \frac{x}{b}}=-\frac{x}{ xa-\arctan (xa)}=-\frac{x}{ xa-xa+\frac13 (xa)^3+o(x^3)}=$$
$$=-\frac{1}{ \frac13 x^2a^3+o(x^2)}\to -\operatorname{sign}(a) \cdot \infty$$
$$\tan\left(ax+\arctan\frac bx\right)=\tan\left(ax+\frac\pi2-\arctan\frac xb\right)=\cot\left(\arctan \frac xb-ax\right) \\\sim\dfrac1{\left(\dfrac1b-a\right)x}$$
and the limit for $ab\ne1$ is $\dfrac b{1-ab}$.
When $ab=1$, by Taylor the argument of the cotangent becomes asymptotic to $-\dfrac{a^3x^3}3$, hence the limit is $\pm\infty$, with the sign of $a$.