Help understand Taylor's theorem - “when” does a function becomes linear?

Question

It is known that for relatively small intervals around some value (say $a$), any (any?) continuous and differentiable function $f$ can be approximated (in the region of the interval) to a linear function via Taylor's theorem with: $f(x) = f(a) + f'(a)*(x-a) + h_k(x)(x-a)$

How is this notion defined more precisely? For example, say that I take the function $x^2$, and I am interested in a region around $x=1$, how can I say how close my function is to linear, as the region around 1 becomes smaller?

Answer 1

Near the value $x = 1$, the function $f(x) = x^2$ is well approximated by the linear function $$ x \mapsto 2x - 1 $$ because $f(1) = 1$ and $f'(1) = 2$.

When you ask "How big an interval is it well-approximated on?" you're asking something like:

"Is $E(x) = x^2 - (2x - 1)$ very different from $0$?" or more generally, for any function $f$ and location $a$,

"Is $E(x) = f(x) - (f(a) + f'(a)(x-a))$ very different from $0$?"

That "error function" can be arbitrarily different from $0$ on any interval you name, i.e., I can find a function with the same value and derivative, but which moves away from the approximating linear function as fast as you like.

But a partial bound is given by knowing that near $a$, $f$ is generally even better approximated by $$f(a) + f'(a) (x - a) + \frac{1}{2} f''(a) (x-a)^2. $$

So if you want the error to be no more than, say, 0.3, then you just choose $a$ so that $\frac{1}{2} f''(a) (x - a)^2$ is no larger than $0.3$. That won't work perfectly, but for many functions it'll come pretty close.

A good example to keep in mind is this:

The special case where the point of approximation is $x = 0$, and the derivative there is $f'(0) = 0$. Then you're asking "If I know that $f(0) = 0$ and $f'(0) = 0$, can I be sure that $|f(x)| < C$ (for some small number $C$) as long as $x$ is within (say) $0.1$ of $0$?

The answer is a resounding "no". Consider the function: $$ f(x) = Ax^2 $$

It has both the properties ($f(0) = f'(0) = 0$) and hence near $0$ is well-approximated by the line $y = 0$. But on the interval $-0.1 < x < 0.1$, the function ranges from $0$ to $\frac{A}{100}$. If $A$ happens to be a million, that's a pretty big range!

Answer 2

Consider the Lagrange Error bound.

For a LINEAR function (which is a polynomial approximation of degree $1$), the error is given by $\displaystyle E\leq\left|\frac{1}{2}\max(|f''(x)|)(x-a)^2\right|$

To generalize, for any degree (n) Taylor approximation (one using derivatives), the error is given by $\displaystyle E \leq \left|\frac{1}{(n+1)!}\max\left(\left|f^{n+1}(x)\right|)(x-a\right)^{n+1}\right|$

Let's say that you want to find the maximum error on $(x_1,x_2)$. Then your $(x-a)$ would be whichever of $x_1$ or $x_2$ is the farthest from $a$. Your $f^{n+1}(x)$ would be the highest value of that derivative on the interval $(x_1,x_2)$.

Also keep in mind that this is the maximum possible error.

Answer 3

A well-known result is:

A function $f$ defined on a neighborhood $N$ of $a \in \mathbb{R}$ is differentiable at $a$, with derivative $d$, if and only if there exists a function $\epsilon$ defined on a neighborhood $N' \subseteq N$ such that $\epsilon(x) \to 0$ as $x \to a$, and for $x \in N'$: $$f(x) = f(a) + d (x - a) + \epsilon(x) (x - a).$$

(As others have indicated, if in fact $f$ is twice differentiable, you can get a somewhat more precise bound on the error based on the error term of a Taylor series approximation.)

Answer 4

We'll make things precise step-by-step.

The first step is pretty simple:

There is a specific (in fact, unique) linear function $l(x)$ which is an arbitrarily good approximation of $x^2$ near $x=1$>

OK, that's not much better, but I wanted to point out the "specific $l$" bit. Now what does it mean to be "an arbitrarily good approximation near a point"?

Well, the point is that $l(x)$ and $x^2$ "go in the same direction" when $x=1$; if you draw the picture of a tangent line to a curve, you'll see what I mean. And "direction" is measured as rise over run. So what we mean is:

There is some linear function $l(x)$ such that (its graph passes through $(1, 1^2)$ and) as $x\rightarrow 1$, the ratios ${l(x)-l(1)\over x-1}$ and ${x^2-1^2\over x-1}$ get closer and closer together.

That's still not precise, but we're almost there. The fully precise definition is:

There is some linear function $l(x)$ such that $$\lim_{x\rightarrow 1}{l(x)-l(1)\over x-1}=\lim_{x\rightarrow 1}{x^2-1^2\over x-1}.$$

Of course this still doesn't finish things off - we need a rigorous definition of limits - but that's something you've probably either seen before or are about to see: it's the "$\epsilon$-$\delta$" stuff. The point is that once we understand what limits are, defining things like "linear approximation" is quick and easy.

You might recognize this as saying, "At $x=1$, $l(x)$ and $x^2$ have the same derivative. This, together with the need to pass through the point $(1, 1^2)$, gives the linear approximation: $$y-1^2=2(x-1)$$ (here the "$2$" coefficient is the derivative of $x^2$ at $x=1$), or $$y=2x-1.$$

Answer 5

I think it is incorrect to ask when does a function become linear. It is like asking when does a circle become a line. A non-linear function will never become linear no matter how much you try. But going by its Taylor's Theorem expansion at point a, every function is linear when you are very very close to a because quadratic and bigger powers of (x-a) are very small. In fact, if x is very close to a, and the function is continuous then every function is (almost) constant near a. And the terms "very small", I think, entirely depends on the accuracy you want in the problem at hand.