You're so close!
You're entirely correct that $U_1 \cap U_2$ is homotopic to $S^1$, so that (writing $X = \mathbb{R}^3 \setminus C$) van kampen's theorem buys us
$$
\pi_1(X) \cong \mathbb{Z}^2 \ast_\mathbb{Z} 1
$$
The question, then, is how to show this amalgamated free product is actually $\mathbb{Z}$. More importantly, we should be able to say what the geometric meaning of this $\mathbb{Z}$ is. After all, it should tell us something about loops in $X$!
The intuitive idea of van kampen's theorem is this:
Loops in $X$ decompose as products of loops in $U_1$ and loops in $U_2$.
After all, if a loop goes through $U_1$ and $U_2$, then it goes through the intersection (where the basepoint is assumed to be) so we can just pinch our loop into one loop in $U_1$ and one loop in $U_2$. In a picture:
we can "pinch" $\gamma$ inside the intersection $U_1 \cap U_2$ in order to break it up into two loops (whose product is homotopic to $\gamma$:
This tells us that every loop in $U_1 \cup U_2$ can be written as a product of loops in $U_1$ and loops in $U_2$. Of course, these loops might not commute! So we're inspired to look at the free product
$\pi_1(U_1) \ast \pi_1(U_2)$.
There's just one hangup: What if there are loops which lie entirely inside $U_1 \cap U_2$? These are the same loop in $X$, but we're "double counting" them, once in $\pi_1(U_1)$ and once in $\pi_1(U_2)$.
We can solve this by taking a quotient to force these loops to be the same! That is, we look at the quotient
$$
\frac{\pi_1(U_1) \ast \pi_1(U_2)}{\iota_1 \gamma = \iota_2 \gamma \text{ for $\gamma \in \pi_1(U_1 \cap U_2)$}}
$$
where $\iota_k$ is the natural maps $\pi_1(U_1 \cap U_2) \to \pi_1(U_k)$ by viewing a loop in $U_1 \cap U_2$ as a loop in the bigger space $U_k$.
This construction comes up surprisingly frequently, so we give it an abbreviated name. Since we're amalgamating the two different "copies" of $\pi_1(U_1 \cap U_2)$, we call this the amalgamated free product
$$\pi_1(U_1) \ast_{\pi_1(U_1 \cap U_2)} \pi_1(U_2)$$
If you want more information about this, see an old answer of mine here.
Concretely, though, how do we work with an amalgamated free product?
Well, the free product $G \ast H$ is the group with generators $g \in G$ and $h \in H$ where we remember how to multiply elements in $G$ and $H$ separately. So in the amalgamated free product, we just impose additional relations saying that $g = h$ whenever $g = \iota_1 \gamma$ and $h = \iota_2 \gamma$.
So for us, we have $\pi_1(U_1)$, which is generated by the two loops $\alpha$ and $\beta$
and of course, $\pi_1(U_2) = 1$, since it's contractible
But (as you've noticed) the intersection is not trivial. Indeed we see $\pi_1 (U_1 \cap U_2)$ is generated by $\beta$.
To compute the amalgamated free product, we need to know the image of this $\beta$ in $\pi_1(U_1)$ and $\pi_1(U_2)$. Notice that, inside of $X$, this is the same loops as what we were calling $\beta$ in $\pi_1(U_1)$. Of course, inside of $U_2$ this loop is contractible, and its image is $1$.
At last, then, we see the amalgamated free product is
$$
\begin{align}
\pi_1(U_1) \ast_{\pi_1(U_1 \cap U_2)} \pi_1(U_2)
&\cong
\langle \alpha, \beta \rangle \ast_{\langle \beta \rangle} \langle 1 \rangle \\
&\cong
\langle \alpha, \beta, 1 \mid \beta = 1 \rangle \\
&\cong
\langle \alpha \rangle \\
&\cong \mathbb{Z}
\end{align}
$$
Which is exactly what we expected!
Moreover, we've learned that the fundamental group of $X$ is generated by $\alpha$, the loop which goes around the copy of $C$ we removed. This is also what we expect geometrically, which is a good sanity check!
I hope this helps ^_^
