7

I have an expression that I want to Taylor Approximate: $$e^{-ax}$$ for some $a$ slightly bigger than $1$. I want to Taylor approximate this in the region close to $x=0$. There are two ways I can do this:

Approximation 1. $e^{-ax}\approx 1-ax$

Here I'm taking $ax$ as the variable, and assuming it is close to $0$.

Approximation 2. $e^{-ax}=(e^{-x})^a\approx (1-x)^a$

Here I'm taking $x$ as the variable, and assuming it is close to $0$.

If we plot this, we see that the second approximation is closer to the truth, and this shouldn't be a surprise, given its functional form:

enter image description here

However, my problem with these two approximations is that I don't have a good feeling of which approach to take for a Taylor approximation for a given problem. For this problem, it turns out the second one is better, but I only know this by plotting it, not by some general principle (and sometimes I'm approximating expressions

So I have a couple of questions:

  1. Is there some general way to know what part of an expression to Taylor approximate?
  2. Also, note that if we taylor approximate the second approximation again, we get the first approximation. Is this a coincidence? If we do two different successive application of Taylor approximation, is it ever possible that we will reach two linear expressions that are nevertheless not equal ?

The last question is especially important, because when I solve a practical problem, and I do multiple approximations to a complex expression, I'd like to know, whether I've found the unique approximation or if there are maybe better ones.

EDIT: Here is an additional question that somewhat relates to my second question. If it is possible to find an exact linear relationship between two variables, will we then always find the exact relationship by Taylor approximations, or is it possible to find an inexact one?

For example, assume we have the equation $x^ay^b=z$, and assume we want to find a linear relationship between the logs of $x,y,z$. We could of course just take the $\ln$ on both sides to get $a\ln x+b\ln y=\ln z$ and be done with it, but assume for a moment that we didn't spot this, and think falsely that the solution has to be approximated.

So we instead denote by $X=\ln x$, and so forth, then we can rewrite the equation as $e^{aX}e^{bY}=e^{Z}$, then, assuming $aX$ and so forth are small, we approximate this by $(1+aX)(1+bY)=1+Z=1+aX+bY+abXY$. If we again approximate this, to get rid of the XY (since we want a linear equation), we get $Z=aX+bY$, i.e. $\ln z=a\ln x +b\ln y$, which is equal to the exact solution.

So again, this begs the question: If it is possible to find an exact linear relationship between two variables, will we then always find the exact relationship by Taylor approximations? Or are there cases when this doesn't work out well? This would be helpful, because it would mean that if we find a linear approximation, we don't have to worry about whether it should have been approximated in the first place.

user56834
  • 12,925
  • In general approximation is a way to approach something that we cannot precise exactly. Based on the fact that the better approximation is the one closer to the precise reality the general principle will be you take the second approximation. Now as for Taylor approximate, the general principle is to take the approximate in the smallest neighbourhood of $0$ since $|ax| \geq |x|$ the smallest neighbourhood of zero is the approach of$x$ near zero which correspond to the second approximate you approach terms without amplification of magnitude $a$. – AlphaXY Dec 03 '17 at 07:07

2 Answers2

6

From the Taylor series

$$ e^{-ax} - (1-ax) = \frac{a^2}{2} x^2 + O(x^3) $$ while $$ e^{-ax} - (1-x)^a = \frac{a}{2} x^2 + O(x^3) $$

So $(1-x)^a$ is a better approximation for small $|x|$ if $|a| > 1$, while $1-ax$ is better for small $|x|$ if $|a|<1$.

On the other hand, much better in both cases would be

$$ \frac{1 - a x - a (1-x)^a}{1-a} $$

because then the error is $O(x^3)$.

Robert Israel
  • 448,999
3

If the function being approximated is analytic, then two distinct paths to the same order approximation will always give the same result. If it is not analytic, then you can construct such a function and choices of paths such that the results will differ.

Given that the answer is either independent of your choice of what to approximate first, or else is non-unique, your approximation technique can safely be based on whatever turns out to be computationally simplest.

Mark Fischler
  • 41,743
  • Thank you Mark! Two questions. Firstly, the first paragraph is exactly what I was looking for! Do you know of a theorem that shows this result? Secondly, I don't really understand the second paragraph. Yes, if the function is analytic, then it is independent of the order of approximation according to your first paragraph, but if it is non-analytic, then the solution is not unique, so why would we then conclude that we can safely do whatever is computationally simplest? If there are multiple solutions, then surely, some of them might be much better than others? – user56834 Dec 03 '17 at 07:10
  • @Programmer2134 Yes, some approximations might be better than others, but since there is no way to tell which is better in those cases, you might as well do the simplest thing. – Mark Fischler Dec 04 '17 at 22:55