Let $f(x)$ is a piecewise function. when $x\ne 3$, $f(x)=x^2$, and when $x=3$, $f(x)=5$. I know that if a function is not continuous, then the function is not differentiable, so $f(x)$ should not be differentiable. But geometrically I think it should be differentiable, because the limit of the slopes of all those tangent lines that via $(3,5)$ is exist.
Could someone tell me where am I wrong？
Others have commented/answered explaining that the limit defining the derivative at $x=3$ doesn't exist, but I think you already knew that. To answer what I see as your real question, I agree with your geometric intuition that the tangent lines to the function around $x = 3$ all exist and their slopes have a limiting value $(6)$. The problem is that the tangent line at $x = 3$ better actually pass through the point $(3, f(3))$, otherwise we can't really call it the ''tangent'' line! This point, $(3, 5)$, is isolated from the rest of the graph of the function. In some sense, it can't "see" the rest of the function. You can think of zooming into this point until the rest of the graph disappears. You are left with a single point. What is a reasonable choice for the slope of the line passing through this point now? Well... anything! We have no information to work with; better to just say the tangent line doesn't exist here.
The definition of the derivative is as follows:
$$f'(x) = \lim_{h\to0}\frac{f(x+h)-f(x)}{h}.$$
In this case:
$$f'(3) = \lim_{h\to0}\frac{(3+h)^2-5}{h} = \lim_{h\to0}\frac{h^2+6h+4}{h},$$
which doesn't exist, since the function $\frac{h^2+6h+4}{h}$ has a vertical assymptote (or just by seeing that the limits from both sides are $\pm\infty$).
