Someone told me this problem long ago and I have made no progress. Let $f:U\to\mathbb{C}$, with $U\subseteq\mathbb{C}$ open, be a continuous function such that $f'(z):=\lim_{z\to z_0}\frac{f(z)-f(z_0)}{z-z_0}$ is defined at every point in $U\setminus A$, where $A$ is a countable subset of $U$. Is $f$ holomorphic?
Due to Riemann's Theorem we can suppose that $A$ has no isolated points. That implies that $A$ is homeomorphic to $\mathbb{Q}$, by Sierpinski´s theorem (concretely, the compact case has been solved as mentioned by Martin R in the comments). I usually imagine the case when $A$ is dense in $U$, that will probably be the toughest one.