Loss of "significance" in $x-y$, $x=0.3721478693, y =0.3720230572$

Question

If we perform the calculations in a computer with a five digit mantissa then I get:

$$\begin{aligned} \mathrm{fl}(x) &=0.37215 \\ \mathrm{fl}(y) &=0.37202 \\ \mathrm{fl}(x)-\mathrm{fl}(y) &=0.00013 \end{aligned}$$

The exact calculation are: $$\begin{aligned} x &=0.3721478693 \\ y &=0.3720230572 \\ x-y &=0.0001248121 \end{aligned}$$

So the relative error will be: $$\left|\frac{x-y-[\mathrm{fl}(x)-\mathrm{fl}(y)]}{x-y}\right|=\left|\frac{0.0001248121-0.00013}{0.0001248121}\right| \approx 4 \%$$

One say that a loss of significance has occurred. But what have been lost? Is it numbers of significance digits in the result $x-y$? If that is correct, can one say that a loss of 5-6 significances digits has been lost? Also, how many have been lost: 5 or 6 significant digits?

Answer 1

If the values $x = 0.3721478693$ and $y = 0.3720230572$ are truly exact values--that is, if $x$ is actually $0.372147869300000\ldots$ (continuing with as many zeros as you could ever want to write) and similarly with $y,$ then $x-y = 0.0001248121$ is an exact result too.

The thing is that in practice we usually have to account for the possibility that our given values of $x$ and $y$ are not exact. In many applications this is the situation we find ourselves in most of the time. If $x$ and $y$ are numbers in a digital computer, we might say they are accurate to within the roundoff error of their last digit, so what we know is that $0.37214786925 \leq x < 0.37214786935.$ Or to put it in a way that doesn't require looking so carefully at so many digits, $x = 0.3721478693 \pm 5\times10^{11}$ and $y = 0.3720230572 \pm 5\times10^{11}$. Hence the most we can say about $x-y$ is that $$ x - y = 0.0001248121 \pm 1\times10^{10}. $$

Two things just happened there. One is that the absolute maximum size of the error has doubled, but the other thing is that the size of the error relative to the answer is much larger than the sizes of the errors relative to the inputs. Before subtraction, you had two numbers on the order of magnitude of $5\times 10^{-1}$ with maximum errors of $5\times 10^{-11}$, a maximum relative error of $10^{-10},$ and afterward you have a number on the order of magnitude of $1\times 10^{-4}$ with a maximum error of $1\times 10^{-10},$ a maximum relative error of $10^{-6}.$

That is, the subtraction cost you four digits of relative precision, most of which can be attributed to the fact that the first three digits of $x$ and $y$ canceled each other out.

---

But let's suppose (for sake of the original exercise, which may have made this assumption) that your ten-decimal-digit values of $x$ and $y$ are exact and that the only source of error is rounding them off to five decimal digits in the computer. You then have $x$ with a relative error

$$ \frac{0.3721478693 - 0.37215}{0.3721478693} \approx -6 \times 10^{-6} $$

and $y$ with a relative error

$$ \frac{0.3720230572- 0.37202}{0.3720230572} \approx 8 \times 10^{-6}, $$

but $x - y$ with a relative error $4 \times 10^{-4}.$ So you have lost almost (but not quite) four decimal digits of precision in the subtraction.

---

Comparing the number of "significant digits" in $0.0001248121$ (infinitely many if $x$ and $y$ really are exact!) versus the number of "significant digits" in $0.00013$ tells you nothing about the loss of significance due to the subtraction. It tells you about the loss of significance due to rounding $x$ and $y$ to just five decimal digits before the subtraction. What happens in the subtraction $0.37215 - 0.37202$ is that you go from numbers with five significant digits to a number with one or two significant digits, depending on whose rules you follow for counting them. (Some would not count the leading $1$ as a significant digit.)

Answer 2

What has been lost is potential significant figures: it could have been seven significant figures if we did it at full precision, but we only got two. Five significant figures have been lost here.