$$
\newcommand{\At}{\tilde{A}}
\newcommand{\Bt}{\tilde{B}}
$$
This looks completely correct to me (but I was wrong; see below. Despite this, almost all of what I wrote was correct, and I leave it here unchanged). I suggest that you now write down a homeomorphism between the two sets. I'll get you started. Let's call the mapping torus $A$, and the other quotient $B$. I'll define a map from $A$ to $B$:
$$
T: A \to B : [ (x, r) ]_A \mapsto [ (x, r) ]_B
$$
where $[]_A$ is the equivalence class of some element $X \times [0, 1]$ in $A$, and similarly for $B$. It's pretty clear that $T$ is continuous, and that $$T \circ \pi_A = \pi_B \circ S,$$ where $\pi_A$ denotes the projection from $X \times [0, 1]$ to $A$, and similarly for $B$, and
$$
S : X \times [0, 1] \to X \times \mathbb R: (x, r) \mapsto (x, r).
$$
So $T$ is the projection of a continuous map (inclusion!) to the quotient; as long as you know that $S$ sends equivalence classes to equivalence classes, the projection of $S$, namely $T$, is well-defined and continuous on the quotient. Proving the statement about equivalence classes should not be too hard for you, but I can show the details if necessary.
Then all you need is a map in the other direction that acts as an inverse to $T$. Try constructing that map as the projection of a map from
$$
X \times \mathbb R \to X \times [0, 1];
$$
I'll bet you can find a good candidate. :)
POST-COMMENT ADDENDUM (in progress):
OP asks for details on why the map $S$ sends equivalence classes to equivalence classes. I need to be clear on the definition here: I mean that if we take two equivalent things in the domain and apply $S$ to them, we'll get two equivalent things in the codomain. There may be some third, or fourth, or other thing in the codomain that's also equivalent to these, but which is not in the image of $S$, and that's OK: we just need the image under $S$ of any equivalence class to be a (possibly proper) subset of some single equivalence class in the codomain.
First, I have to slightly correct my earlier assertion that the action was the correct one. It should instead have
$$
n \cdot (x, r) := (f^n(x), r-n),
$$
for this gives
$$
1 \cdot (x, 1) := (f(x), 0),
$$
i.e., it ends up identifying $(x, 1)$ with $(f(x), 0)$; the original action you defined had a sign-error.
I'm going to use
$\At$ to denote $X \times [0, 1]$, and similarly use $\Bt$ for $X \times \mathbb R$, so that $\pi_A$ sends $\At$ to $A$, and similarly for $\pi_B$. So now $S$ is a map from $\At$ to $\Bt$.
An equivalence class in $\At$ is either $\{(x, r) \}$, where $r$ is neither $0$ nor $1$, or it's a pair $\{ (x, 1), (f(x), 0) \}$. In the first case, we have
$$
S(\{(x, r) \}) = \{S(x, r) \} = \{ (x, r) \} \in \Bt.
$$
That second set (which has only one element) is entirely contained in a single equivalence class of $\Bt$.
What about in the other case?
\begin{align}
S(\{ (x, 1), (f(x), 0) \}) &= \{ S(x, 1), S(f(x), 0) \} \\
&= \{ (x, 1), (f(x), 0) \}
\end{align}
The latter is a (proper) subset of an equivalence class of $\Bt$ as well -- the equivalence class also contains $(f(f(x)), -1)$, for instance -- so $S$ does indeed take equivalence classes to equivalence classes in the sense described above.
We now need to define a map $F$ from $\Bt$ to $\At$ that acts as an inverse to $S$, and to show that if $(x, a)$ and $(y, b)$ are equivalent in $\Bt$, then $F(x, a)$ and $F(y, b)$ are equivalent in $\At$.
Well, if $(x,a)$ and $(y, b)$ are equivalent, then there's an integer $n$ with $n \cdot (x, a) = (y, b)$, which means that $b = a - n$, so we've really got $(x, a)$ and $(y, a - n)$ being equivalent. But that means that $y = f^n(x)$. That means that our map $F$, when we construct it, must have two properties:
(1) For $0 < a < 1$, $F(x, a) = (x, a)$, so it'll be an inverse to $S$, and
(2) For any $n$, F(x, a) = F(f^n(x), a-n)$.
The only thing left uncertain by these two definitions is whether $F(x, 1) = (x, 1)$ or $F(x, 1) = (f(x), 0)$. I'll choose the latter convention.
The unique function satisfying these two conditions is
$$
F(x, a) = (f^{-\lfloor a \rfloor}(x), a \bmod 1).
$$
So we need to show this:
Given $(x, a)$ and $(f^n(x), a - n)$ (i.e., two equivalent points in $\Bt$), the points
$F(x, a)$ and $F(f^n(x), a - n)$ are equivalent in $\At$.
Working from the definition, we have
\begin{align}
F(f^n(x), a-n)
&= (f^{-\lfloor a-n \rfloor}(f^n(x)), (a-n) \bmod 1)\\
&= (f^{-\lfloor a \rfloor -n}(f^n(x)), (a-n) \bmod 1)\\
&= (f^{-\lfloor a \rfloor }(f^{-n}(f^n(x))), (a-n) \bmod 1)\\
&= (f^{-\lfloor a \rfloor }(x), a \bmod 1)\\
&= F(x, a).
\end{align}
And that completes the construction of $F$ and the proof that it takes equivalence classes to equivalence classes, I believe.
