The statement is true in the sense, that $F'$ is continuous at $x_0$ on its domain, which need not to be the entire interval $(a, b)$. We may drop that $f$ is differentiable at $x_0$, we may also replace Riemann integrability with Lebesgue.
Claim:
Let $D = \{ x\in (a, b) \mid F \text{ is differentiable at } x \}$. For $x_n\in D$ with $x_n\to x_0$ it follows $F'(x_n) \to F'(x_0) = f(x_0)$.
Proof:
Notice that as $f$ is continuous at $x_0$ it follows
$$F'(x_0) = \lim_{y\to x_0} \frac{1}{y - x_0} \int_{x_0}^y f(t) dt = f(x_0).$$
Fix $\varepsilon > 0$.
- As $f$ is continuous at $x_0$, a $\delta > 0$ exists such that for every $x\in [a, b] \cap (x_0 - \delta, x_0 + \delta)$ it follows
$$|f(x) - f(x_0)| < \frac\varepsilon2.$$
- As $x_n\to x_0$, a $N> 0$ exists such that for every $n \ge N$ it follows $$|x_n - x_0| < \frac{\delta}{2}.$$
- As $F$ is differentiable at $x_n$, a $\eta_n > 0$ exists such that for every $y\in [a, b] \cap (x_n - \eta_n, x_n + \eta_n)$ with $y\ne x_n$ it follows
$$ \left|\frac{F(y) - F(x_n)}{y - x_n} - F'(x_n) \right| < \frac\varepsilon2. $$
Putting these together, for every $y$ with $|y - x_n| < \min(\eta_n, \frac{\delta_n}{2})$ it follows
\begin{align*}
\left| F'(x_n) - f(x_0) \right|
&\le \left| F'(x_n) - \frac{F(y) - F(x_n)}{y - x_n} \right| + \left| \frac{F(y) - F(x_n)}{y - x_n} - f(x_0) \right| \\
& \le \frac\varepsilon 2 + \left| \frac{1}{y - x_n}\int_{x_n}^y f(t) - f(x_0) dt \right| \\
& \le\frac\varepsilon 2 + \frac{1}{y - x_n}\int_{x_n}^y \underbrace{|f(t) - f(x_0)|}_{\le \varepsilon / 2} dt \\
&\le \epsilon,
\end{align*}
as $|t - x_0| \le |t - x_n| + |x_n - x_0| \le |y - x_n| + |x_n - x_0| < \delta$.
That is, $F'(x_n)$ converges to $f(x_0) = F'(x_0)$.
Notes:
If $f$ is differentiable at $x_0$, a similar estimation shows that
$$\lim_{n\to\infty} \frac{F'(x_n) - F'(x_0)}{x_n - x_0} = f'(x_0)$$
Old Answer for reference only:
This rather an extensive comment than an answer.
Since $f$ is Riemann integrable, its set of continuity
$$ C = \left\{ x\in[a,b] \mid \lim_{t\to x} f(t) = f(x) \right\} $$
has full measure, that is $|C| = b-a$. Thus, $F'$ exists on $C$ and agrees with $f$ on $C$. Trivially, $F'$ restricted onto $C$ is continuous.
That says nothing about the domain of $F'$, which might be larger than $C$, and whether $F'$ is continuous at $x_0$.