Assume $ab \ne 0$. If you differentiate $$F(x;a,b) = \frac{1}{ab} \tan^{-1} \left( \frac{a}{b} \tan x \right),$$ you get $$f(x;a,b) = \frac{1}{ab} \left(\frac{1}{1 + \frac{a^2}{b^2} \tan^2 x}\right) \cdot \frac{a}{b} \sec^2 x = \frac{\sec^2 x}{b^2 + a^2 \tan^2 x} = \frac{1}{a^2 \sin^2 x + b^2 \cos^2 x},$$ so at least this works for all $x$ for which $F(x)$ is well-defined; however note $F(\pi/2;a,b)$ is not such a point, and this point is located in the interval $(0,\pi)$. What happens in this case? We know $f$ is periodic with period $\pi$ and is smooth for all reals. The issue is that the inverse tangent in $F$ admits any number of possible angles for which $$\tan (ab F(x;a,b)) = \frac{a}{b} \tan x,$$ each differing by some integer multiple of $\pi$. So it is possible to choose an antiderivative whose behavior is continuous on $(0,\pi)$.
For example: what about $$F^*(x;a,b) = \frac{1}{ab} \tan^{-1} \left(\frac{b}{a} \tan \left( x - \frac{\pi}{2} \right)\right)?$$ If you differentiate this function, it also evaluates to $f$. But we have no issue with discontinuity, since when $x \in (0,\pi)$, $\tan (x - \pi/2) \in (-\infty, \infty)$ and we have effectively "hidden" the discontinuity at the endpoints of the interval of integration. The reality, of course, is that it is possible to define a strictly increasing function $G(x;a,b)$ that is smooth everywhere just like $f$, whose derivative equals $f$ everywhere; we just have to be careful about how we specify the evaluation of the inverse trigonometric function.
It bears mentioning that moving the discontinuity to the endpoints of the interval of integration doesn't completely address the issue of evaluating the definite integral, since now you have an improper integral. It does, however, help us to understand how the definite integral cannot be zero, and why it evaluated to zero when we used $F$ at the endpoints.