The circle has center $(h,1)$, and radius $r = 1$, and there is a point $(x_0,y_0)$ on the circle that satisfies $y_0 = x_0^2$; thus
$$(x_0 - h)^2 + (x_0^2 - 1)^2 = 1. \tag{1}$$
However, there is another constraint: at the point of tangency, the slope of the tangent line to the parabola and the circle are the same. The slope of $y = x^2$ at $x = x_0$ is $$\left[\frac{dy}{dx}\right]_{x=x_0} = 2x_0. \tag{2}$$ Via implicit differentiation, we have $$2(x - h) + 2(y-1)\frac{dy}{dx} = 0,$$ or $$\frac{dy}{dx} = \frac{x-h}{1-y},$$ hence $$\left[\frac{dy}{dx}\right]_{(x,y) = (x_0, x_0^2)} = \frac{x_0 - h}{1 - x_0^2}. \tag{3}$$ As these two slopes must be equal, we have from Equations $(2)$ and $(3)$ the condition $$2x_0(1-x_0^2) = x_0 - h. \tag{4}$$ Eliminating $x_0$ from the system of equations in $(1)$ and $(4)$ yields $$16 h^4 - 71 h^2 + 2 = 0,$$ where we have assumed $h \ne 0$. This quadratic in $h^2$ has the solution set $$h \in \left\{ \pm \frac{1}{4}\sqrt{\frac{71 \pm \sqrt{17}}{2}} \right\}.$$ Of these, the outermost sign choice merely corresponds to the axis of symmetry of the parabola, but the inner sign choice affects the nature of tangency; for the negative sign choice, the circle is internally tangent to the parabola and exhibits intersections with other points of the parabola. If we exclude these, then the only solutions correspond to the positive choice of sign, i.e. $$h \in \pm \frac{1}{4} \sqrt{\frac{71 + \sqrt{17}}{2}} \approx \pm 2.0998.$$