Why is $(1+x)^2$ less accurate than $ (x+2)x+1$ for small $x$?

Question

I've known that accuracy is based on the amount of roundings (or multiplications) that occur, but from what I can tell, both equations will require the same amount.

My first thought was to related $(1+x)^2$ to $1+2x+x^2$ and say that there are two mandatory roundings that happen, but that equation is also equivalent to $(x+2)x+1$, so it wouldn't make sense to make my standard point off of that.

Any help will be greatly appreciated, thank you.

Answer 1

Let's us assume that $x \approx 10^{-(n-k)}$, where $n$ is the number of significant digits remembered (in double arithmetic, this is roughly $16$) and $k$ is a small positive integer. Now, compare:

• $1+x$ has only $k$ significant digits from $x$.

• $x+2$ has only $k$ significant digits from $x$.

Not really different. But, lets go on:

• $(1+x)^2$ is a square of a number that has only $k$ significant digits from $x$, so only these $k$ digits matter.

• $(x+2)x$ is a number with only $k$ significant digits from $x$ multiplied by the whole $x$ (with multiplication, you don't lose significant digits), so the whole of $x$ matters (not completely, of course, but to a certain point; remember that a good part of it - $n-k$ digits - was lost in $x+2$).

Now, for the final "+1":

• $(x+2)x + 1$ has lost many digits again. However, if $(x+2)x - 2$ has more significant digits than $x$, (for example, if the first significant digit of $x$ is $4$ or more), than more of $x$ has survived. More precisely, in that case, $k+1$ digits of $(x+2)x$ have survived, and this additional digit has some impact from $x$, as explained above.

As an example, here is some C code:

#include <stdio.h>

int main() {
  double x = 0.000000000000654, t;
  t = 1+x;
  t *= t;
  printf("(1+x)^2  = %.16f\n", t);
  t = x+2;
  t *= x;
  t += 1;
  printf("(x+2)x+1 = %.16f\n", t);
  t = x*x;
  t += 2*x;
  t += 1;
  printf("x^2+2x+1 = %.16f\n", t);
  t = 1;
  t += 2*x;
  t += x*x;
  printf("1+2x+x^2 = %.16f\n", t);
  return 0;
}

Results:

(1+x)^2  = 1.0000000000013078
(x+2)x+1 = 1.0000000000013081
x^2+2x+1 = 1.0000000000013081
1+2x+x^2 = 1.0000000000013081

In Mathematica, computed with the $20$ digit precision:

x = 654/10^16;
N[(x + 1)^2, 20]

The result:

1.0000000000001308000

Note that $2 \cdot 654 = 1308$. Computed with the $33$ digit precision, the above yields

1.00000000000013080000000000427716

so we have obviously computed the $2x$ part with better precision in the last $3$ expressions (in my C code) than in the first one.

Answer 2

A heuristic explanation: suppose that $x$ is very small; then the floating point approximation to $1+x$ will often be erroneous because of 'float rounding': for instance, suppose that you have 10 digits of core 'float' accuracy and that $x\approx 10^{-8}$; then $1+x$ will lose all but two digits of $x$'s accuracy, and when we square $1+x$ then the remaining digits stay lost. By contrast, when adding $2+x$ we may lose another bit of accuracy initially, but the result is then multiplied by $x$ and this multiplication gains the 'full accuracy' of its second $x$ factor.

(On the other hand, the fact that $1$ is then added to the result leaves me skeptical, because much of that accuracy is promptly lost again; I would like this explanation much more if it were comparing e.g. $(x+2)x$ vs. $(x+1)^2-1$.)

Answer 3

In interval arithmetic, this question is answered differently than you may expect. In particular, the accuracy of every formula can be rigorously proven directly by evaluating increasingly small intervals up to the accuracy limit of the system.

Here is an example of how it works:

Given $x=[a,b]$ as the interval being evaluated with $a\lt b$, evaluate the following:

$$f_1(x)=(x+1)^2$$

$$f_2(x)=(x+2)x+1$$

$$f_3(x)=x^2+2x+1$$

$$f_1([a,b])=([a,b]+1)^2=[a+1,b+1]^2=[\min((a+1)^2,(b+1)^2),\max((a+1)^2,(b+1)^2)]$$

$$f_2([a,b])=([a,b]+2)[a,b]+1=[a+2,b+2][a,b]+1=[\min(a(a+2),a(b+2),b(a+2),b(b+2))+1,\max(a(a+2),a(b+2),b(a+2),b(b+2))+1]$$

$$f_3([a,b])=[a,b]^2+2[a,b]+1=[\min(a^2,b^2),\max(a^2,b^2)]+[2a,2b]+1=[\min(a^2,b^2)+2a+1,\max(a^2,b^2)+2b+1]$$

Given that we know only that $a\lt b$, $f_1$ and $f_3$ demonstrate the greatest similarity, but it should be obvious that if $|a|\gt|b|$ then $f_1\ne f_3$. If we say that a measure of the accuracy is the ratio of the length of the input interval compared to the length of the output interval, then $f_1$ has greater accuracy than $f_3$ overall.

This isn't the best possible demonstration of how interval arithmetic works, but it should show how intervals can be used in evaluating functions. In particular, it is clear that the accuracy is the opposite of that demonstrated in other answers here.