Why do implicit numeric methods have superior stability properties than explicit ones?

Question

I've been taking classes in numerical methods and we were told that implicit methods are more stable than explicit ones. This is very easy to show with some differential equations, but I dont understand why this would be a general phenomenon. I was searching online but did not find any satisfying answer. Is this an intrinsic property of all implicit methods? Are there examples where the explicit method has better stability compared to the corresponding implicit one?

Answer 1

An explicit method for $x'(t)=f(t,x(t))$ uses its stages to construct Taylor terms of the expansion of $x(t_n+h)$, $x(t_n)=x_n$, in $h$ with the correct coefficients for the step to the next point. But apart from satisfying the order conditions to get the correct coefficients for the Taylor expansion in the basis point $x_n$, nothing ties the computed values along the step from $x_n$ to $x_{n+1}$ to the given ODE.

In an implicit step, the ODE is satisfied as equation in at least one other point. In a most visible fashion this can be observed for collocation methods. The intermediate values of the method step can there be encoded in a polynomial $p(s)$ that (conceptually) approximates $x(t_n+sh)$, with $p(0)=x_n$. The method then is described by values $c_1,...,c_q\in[0,1]$ where $$p'(c_k)=hf(t_n+c_kh,\,p(c_k)),~~k=1,..,q$$ is demanded to hold. Many of the named implicit methods can be constructed this way. With $q=1$, $c_1=\frac12$ you get the midpoint method, with $q=2$, $c_1=0$, $c_2=1$ the trapezoidal method, with $c_{1,2}=\frac12\pm\frac{\sqrt3}6$ the 4th order Gauss-2 method, with $q=3$, $c_{1,2}=\frac25\pm\frac{\sqrt{6}}{10}$, $c_3=1$ a Radau IIA method,...

So as the implicit method includes more "feed-back" from the numerical solution path, it is in general more stable than an explicit method.

But note that an implicit method with a fixed procedure to solve its implicit system, that is, a fixed number of approximation steps of a fixed type, is again an explicit method. It can then be compared to other explicit methods that use the same number of function evaluations of the ODE right-hand-side function. This can be by using a higher number of stages to get a higher method order or satisfy additional conditions reducing the step error. Or by replacing one step of the implicit method with multiple steps with a reduced step size of an explicit method.

Thus to be truly an implicit method its implementation has to be free to iterate its implicit solver until the error in the implicit system is small enough, typically an order of magnitude or power in the step size smaller than the truncation error of the method step.

Answer 2

Let us consider a vector differential equation of the form $$ \frac{\text{d}{\bf{x}}}{\text{d}t}={\bf{f}}({\bf{x}}),~~{\bf{x}}={\bf{x}}(t),~~{\bf{x}}(0)={\bf{x}}_0. $$ The new (unknown) time level is denoted by superscript $n$ and the old (known) time level is denoted by superscript $n-1$. Consider a scheme $$ \frac{{\bf{x}}^n-{\bf{x}}^{n-1}}{\triangle t}=\theta{\bf{f}}({\bf{x}}^n)+(1-\theta){\bf{f}}({\bf{x}}^{n-1}). $$ The scheme is implicit for all values $\theta \in (0,1]$ and only for $\theta=0$ you get an explicit scheme (the explicit Euler scheme). I offer two intuitive perspectives. One is that the most correct value seems to be the middle value, i.e. $\theta=\frac{1}{2}$, which however yields an implicit scheme. Second perspective: in the explicit scheme you get the simplest possible linear problem, which is linear also if ${\bf{f}}$ is nonlinear. When you pay less, you also get less (in numerics it is often so). Consequently you get a scheme with reduced stability.

As an example consider explicit vs implicit Runge-Kutta methods. Implicit ones are numerically demanding, but you also get excellent numerical properties, which are needed when one solves a difficult class of equations (the so-called stiff problems for example).