3

In order to have precision of 5 decimal points in a Numerical Analysis method for the solution of an equation then:

$$|x_{n+1}-\rho|\leq \frac{1}{2}\cdot 10^{5}$$

where $x_{n+1}$ the current approximation of the solution and $\rho$ the actual solution.

I am reading though in a textbook that if you want to have precision of 5 decimal points then your condition to stop the method is:

$$|x_{n+1}-x_{n}|\leq \frac{1}{2}\cdot 10^{5}$$

where $x_{n}$ is approximation of the solution in the previous step.

What I am thinking is that if the method is indeed converging to the actual solution then you will find the correct approximation using the second check, but if the method converges in a false point (i.e. the method doesn't work correctly for a number of reasons) then you will have found a wrong approximation (but hey you will at least have it with a precision of 5 decimal points).

Isn't this right? How can the second check be used generally?

Lutz Lehmann
  • 126,666
Adam
  • 351
  • When you finish the loop you need to use the value you obtained to check that it produces the required accuracy by substituting it in the original function. If it produces the correct result, then it is an acceptable approximation of the root. – NoChance Feb 10 '16 at 21:17
  • @NoChance This is kind of a trial and error sometimes. Isn't any rule or a reason why this is correct? – Adam Feb 11 '16 at 01:01
  • I really don't know. To a great extent, the result depends on the function in question as well as your choice of how to express $x_{n+1}=g(x_n)$. – NoChance Feb 11 '16 at 02:29
  • @NoChance Don't worry your comment helped me because from what I see is that we are always trying to find complex answers in easy questions so your input is really useful when you have to check the results of a method. – Adam Feb 11 '16 at 16:13

1 Answers1

2

Consider the problem of solving the equation $f(x) = 0$, where $f : \mathbb{R} \rightarrow \mathbb{R}$ is at least continuous.

In general, it root finder should monitor both the distance between successive iterates, i.e. \begin{equation} \delta_n = |x_n-x_{n+1}| \end{equation} as well as the absolute value of the residual, i.e. \begin{equation} \epsilon_n = |f(x_n)| \end{equation} and terminate if one of these numbers drops below a user defined threshold. Some applications do not require a small error and the user is happy if the residual is small.

If the iteration is converging to a root, then both $\delta_n$ and $\epsilon_n$ will eventually be small.

However, and this is critical, the reverse is not true! Given thresholds $\delta$ and $\epsilon$ it is possible to construct a function $f$ which has a well defined zero for which Newton's method diverges and satisfies \begin{equation} \delta_n \leq \delta \quad\text{and}\quad \epsilon_n < \epsilon \end{equation} for all sufficiently large $n$. A specific example is the function \begin{equation} f_{\lambda}(x) = p(x) \exp(-\lambda x) \end{equation} where $p$ is a real polynomial with a single root $r$ and $\lambda > 0$ is a real number to be determined shortly. Obviously, we would be mad not to focus on $p(x)$, but while it is easy for a human to make that determination it is hard to infuse a machine with this ability. Away from the root $r$, Newton's method takes the form \begin{multline} x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} = x_n - \frac{f(x_n)}{p'(x_n)e^{-\lambda x_n} - \lambda f(x_n)} \\ = x_n - \frac{1}{p'(x_n)/p(x) - \lambda} \approx x_n + \frac{1}{\lambda}, \end{multline} because $\frac{p'(x)}{p(x)} \rightarrow 0$ as $x \rightarrow \infty$. We conclude that if the user has picked an initial guess sufficient far to the right of the the root, \begin{equation} r \ll x_0 \end{equation} then not only will we continue to move away from the root as \begin{equation} r < x_n < x_{n+1} \sim x_n + \frac{1}{\lambda} \end{equation} but we will have \begin{equation} \delta_n < \delta \end{equation} provided $\lambda > \frac{1}{\delta}$. Moreover, since $x_n \rightarrow \infty$ we will eventually have \begin{equation} \epsilon_n < \epsilon \end{equation} In short, if a routine is only monitoring the distance between successive iterates and the size of the residual, then it can be defeated by a sufficiently clumsy user. An alert user will examine the entire sequence of approximations and discover that it is not converging towards a real number.

A robust routine will instead strive to maintain and shrink a bracket around the root, i.e. an interval $[a,b]$ for which $f(a)$ and $f(b)$ have different sign. As input, it should accept an initial bracket as well as handler to a function which computes not only a good approximation $\hat{y}$ of $y = f(x)$, but also an error bound $|y-\hat{y}| \leq m u$, where $m = m(y)$ is an integer and $u$ is the unit round off error. There is no point in continuing past the point where $|\hat{y}| \leq |y -\hat{y}|$ as the true value of might as well be zero. Computing an error bound of this type allows the routine to make that determination.

If you successfully shrink the length of the bracket down to $10^{-5}$, then the midpoint, i.e. $\mu = a + \frac{b-a}{2}$ will approximate the root with 5 decimal figures.

A good routine can be written by merging the bisection method (bracket) with the secant method (fast local convergence). Bounds for the difference between the true value of $y = f(x)$ and the computed value $\hat{y} = f(x)$ are harder to come by but can be established using the method of running error bounds. N. Higham derives a running error bound for polynomials discusses in his book "Accuracy and Stability of Numerical algorithms".

Carl Christian
  • 12,583
  • 1
  • 14
  • 37
  • Thank you for your well-written answer. If I understand correctly then you are saying that if a method is proven to converge to a root then both checks are good but if not then the distance between successive iterations may grow bigger every time? One more thing that I don't understand is "If $\lambda>0$, then not only are we marching further away from the root, but if $\lambda$ is large, then $\delta_{n}$ is small and given enough time $\epsilon_{n}$ will also be small." If $\lambda$ is growing how can $\epsilon_{n}$ become small? – Adam Feb 11 '16 at 16:21
  • @Adam: I thought it over and decided to extend a part of the answer. I think that it is clearer now and that it should take care of your questions. Otherwise, let me know. – Carl Christian Feb 11 '16 at 18:04
  • Your answer is great. Thank you very much for your explanation. You added even more than I asked for. – Adam Feb 11 '16 at 18:48
  • @Adam: Thank you very much. Answering questions and modifying the explanation to clarify key points helps my own understanding. – Carl Christian Feb 11 '16 at 19:00
  • A minor note for the ending explanation: it is not actually necessary to use the approximated error $|x_n-x_{n+1}|$, as the more reliable $|a-b|$ may be used. Guaranteed convergence of the bracket is usually done by shifting the guessed root towards the midpoint of the interval by half the desired error. This generally causes convergence in only one more iteration. – Simply Beautiful Art Sep 15 '20 at 01:26
  • @SimplyBeautifulArt I have added words to emphasize the difference between the two types of error which are needed here. The error $x-x_n$ which the user cares about and the error $y-\hat{y}$ which the routine needs to bound in order to determine if the current bracket has a sign change or if the iteration should be terminated. – Carl Christian Sep 15 '20 at 08:41
  • This is fine, but I was just pointing out that in the case of bracketing methods, the approximated error $\delta_n=|x_n-x_{n+1}|$ is less reliable than using $|a-b|$, the size of the bracketing interval. – Simply Beautiful Art Sep 15 '20 at 15:58
  • @SimplyBeautifulArt: I now understand what you meant and I agree that one should monitor the width of the bracket rather than the difference between two successive approximations. Indeed, this was the motivation for my second to last paragraph which tied directly to OP's question. – Carl Christian Sep 15 '20 at 20:15