1. If you are aware of the Big-O notation, then using the Taylor's Theorem, we get
$$\sqrt{1+\varepsilon} = 1 + \frac{1}{2}\varepsilon + \mathcal{O}(\varepsilon^2) \quad \text{as} \quad \varepsilon \to 0.$$
As such,
$$ \sqrt{x+a} = \sqrt{x} \sqrt{1 + \frac{a}{x}} = \sqrt{x} + \frac{a}{2\sqrt{x}} + \mathcal{O}\left(\frac{1}{x^{3/2}}\right) \quad \text{as} \quad x \to \infty, $$
and similarly
$$ \sqrt{x-b} = \sqrt{x} - \frac{b}{2\sqrt{x}} + \mathcal{O}\left(\frac{1}{x^{3/2}}\right) \quad \text{as} \quad x \to \infty. $$
Plugging this back and applying the same estimate again,
$$ \sqrt{\sqrt{x+a} - \sqrt{x}}
= \sqrt{\frac{a}{2\sqrt{x}} + \mathcal{O}\left(\frac{1}{x^{3/2}}\right)}
= \sqrt{\frac{a}{2}} \frac{1}{x^{1/4}} + \mathcal{O}\left(\frac{1}{x^{5/4}}\right) $$
and
$$ \sqrt{\sqrt{x} - \sqrt{x-b}}
= \sqrt{\frac{b}{2}} \frac{1}{x^{1/4}} + \mathcal{O}\left(\frac{1}{x^{5/4}}\right). $$
Plugging this back, the integrand is now asymptotically
$$ \left( \sqrt{\frac{a}{2}} - \sqrt{\frac{b}{2}} \right) \frac{1}{x^{1/4}} + \mathcal{O}\left(\frac{1}{x^{5/4}}\right), $$
and for the integral of this over $[b, \infty)$ to converge, the first term must vanish. (The error term $\mathcal{O}(x^{-5/4})$ does not pose any issue to the integrability, because we know that $\int_{b}^{\infty} x^{-5/4} \, \mathrm{d}x$ is finite.) Therefore the integral converges if and only if $a = b$.
2. If you are not used to analyzing the asymptotic behavior of a function, then so called "rationalization" technique will be helpful:
\begin{align*}
\sqrt{x+a} - \sqrt{x}
= (\sqrt{x+a} - \sqrt{x}) \cdot \frac{\sqrt{x+a} + \sqrt{x}}{\sqrt{x+a} + \sqrt{x}}
= \frac{a}{\sqrt{x+a} + \sqrt{x}},
\end{align*}
and similarly we have
$$ \sqrt{x}-\sqrt{x-b} = \frac{b}{\sqrt{x} + \sqrt{x-b}}. $$
Plugging this back,
\begin{align*}
&\sqrt{\sqrt{x+a} - \sqrt{x}} - \sqrt{\sqrt{x}-\sqrt{x-b}} \\
&= \frac{\sqrt{a}}{\sqrt{\sqrt{x+a} + \sqrt{x}}} - \frac{\sqrt{b}}{\sqrt{\sqrt{x} + \sqrt{x-b}}} \\
&= \frac{\sqrt{a \left(\sqrt{x} + \sqrt{x-b}\right)} - \sqrt{b\left(\sqrt{x+a} + \sqrt{x}\right)}}{\sqrt{\left(\sqrt{x+a} + \sqrt{x}\right)\left(\sqrt{x} + \sqrt{x-b}\right)}} \\
&= \frac{a\left(\sqrt{x} + \sqrt{x-b}\right) - b\left(\sqrt{x+a} + \sqrt{x}\right)}{\sqrt{\left(\sqrt{x+a} + \sqrt{x}\right)\left(\sqrt{x} + \sqrt{x-b}\right)}\left(\sqrt{a \left(\sqrt{x} + \sqrt{x-b}\right)} + \sqrt{b\left(\sqrt{x+a} + \sqrt{x}\right)}\right)}.
\end{align*}
If $a = b$, then the numerator becomes
$$ a \left(\sqrt{x+a} - \sqrt{x-a}\right) = \frac{2a^2}{\sqrt{x+a} + \sqrt{x-a}} \sim \frac{a^2}{\sqrt{x}} \quad \text{as} \quad x \to \infty. $$
If $a \neq b$, then the numerator is $\sim 2(a - b)\sqrt{x} $.
In any cases, the denominator is $\sim 2\sqrt{2}\left(\sqrt{a} + \sqrt{b} \right) x^{3/4} $.
Combining these two observations, we know that the integrand is of order $x^{-1/4}$ if $ a\neq b$ and of order $x^{-5/4}$ if $a = b$. So the integral converges if and only if $ a = b$.
FullSimplify[ Series[Sqrt[Sqrt[x + a] - Sqrt[x]] - Sqrt[-Sqrt[x - b] + Sqrt[x]], {x, Infinity, 2}], Assumptions -> x > 0 && {a, b} \[Element] Reals]produces $$\frac{\sqrt[4]{\frac{1}{x}} \left(\sqrt{a}-\sqrt{b}\right)}{\sqrt{2}}-\frac{\left(\frac{1}{x}\right)^{5/4} \left(a^{3/2}+b^{3/2}\right)}{8 \sqrt{2}}+O\left(\left(\frac{1}{x}\right)^{9/4}\right) .$$ – user64494 Aug 04 '21 at 05:48