Yes. In fact if $A$ is a Banach algebra with identity and $G$ is the group of invertible elements then the map $x\to x^{-1}$ is continuous on $G$:
Say $x\in G$. If $||y||<1/||x^{-1}||$ then $||yx^{-1}||<1$, hence $x-y=(e-yx^{-1})x\in G$ and in fact $$(x-y)^{-1}=((e-yx^{-1})x)^{-1}=x^{-1}(e-yx^{-1})^{-1}
=x^{-1}(e+yx^{-1}+(yx^{-1})^2+\dots).$$This shows that $(x-y)^{-1}\to x^{-1}$ as $y\to 0$:
$$||(x-y)^{-1}-x^{-1}||\le||x^{-1}||(||x^{-1}||(||y||\,||x^{-1}||+||y||^2||x^{-1}||^2+\dots).$$
(If you don't know anything about Banach algebras you can find an answer to your question above by assuming $x$ and $y$ are operators and $e=I$. Note that we use the fact that $||xy||\le||x||\,||y||$ in various places.)