Taylor Expansion on Newton's Method

Question

I get that the Newton's Method is can be derive from the first Taylor Expansion, which is $$ f(r) = f(x_{0})+f'(x_{0})(r - x_{0}) + \frac{f''(\xi)}{2}(r - x_{0})^{2} $$ with $r$ is the root of $f(x)$, the question is, why only takes first term of Taylor, not until the second term, or greater?

Answer 1

1. If you use higher order terms of the Taylor polynomial and try to derive Newton's method as in the linear case, then you have to solve an $n\geq 2$ polynomial each iteration. This tags on an additional root finding problem to every iteration, to what was originally a root finding problem.

2. Abstractly, most $C^n$ functions are not $C^{n+1}$.

3. If the function you are working with is an interpolation of some data (such as splines), you may not be able to gather appropriate data to determine what the derivatives higher than 1 or 2 should look like. Furthermore, raising the degree of polynomial splines tends to amplify the error.

The 2nd degree case:

Let $r$ be a root of $f$. $$0=f(r)\approx f(x_0)+f^{\prime}(x_0)(r-x_0)+\frac{f^{(2)}(x_0)}{2}(r-x_0)^2$$ We want to solve this for $r$. The quadratic formula gives $$r= x_0+\frac{-f^{\prime}(x_0)+\sqrt{(f^{\prime}(x_0))^2-2f(x_0)f^{(2)}(x_0)}}{f^{(2)}(x_0)}$$

So the naive degree 2 Newton algorithm would be:

$$x_{i+1}= x_i+\frac{-f^{\prime}(x_i)+\sqrt{(f^{\prime}(x_i))^2-2f(x_i)f^{(2)}(x_i)}}{f^{(2)}(x_i)}$$

So you have to calculate a square root (using another algorithm) every iteration, and there is an issue of which square root to take (may have to be complex). Furthermore, if you want to use the Taylor polynomial of degree $\geq 5$, there is no algebraic simplification of the polynomial. So you will have to use a polynomial root finding algorithm each iteration. There are other root finding algorithms (e.g. Householder's method) and some people have developed generalizations of Newton's method to higher degrees, but I cannot find evidence that these generalizations are useful enough to replace it.