2

I was asked to numerically calculate Bessel functions for certain points and report their values to "6 decimal places of exactness". I did this in matlab and there's no truncate function, so I was forced to round the number.

However this got me thinking what 6 decimal places of exactness means. Does it mean I should round the sixth decimal place or not? Is there any mathematical convention as to what that sentence should mean?

By the way I looked up $\pi$ to three decimal places and people report 3.142, not 3.141.

I have no way of asking my professor.

Thanks.

ben ari
  • 365

2 Answers2

5

There are basically two seemingly reasonable possibilities.

  1. The $n$th decimal digit and all preceding digits are correct. Then the error is at most $10^{-n}$. But this is not sufficient. For instance, if the true number is $0.199$, then your error bound must be less than $0.001$ to ensure that the first decimal digit is correct.

  2. The $n$th decimal digit is where the first rounding occurs. Then the error is at most $10^{-n}/2$, and this condition is sufficient. In this case it might be that none of the digits are correct. For instance, $0.200$ is an "accurate to two decimal places" approximation of $0.199$ in this sense.

The second option is much easier to check and gives you much clearer information about the actual size of the error, which is why it is the better way to measure these sorts of things in practice. On the other hand, the first option is "prettier"; we have to follow the first option in order to really be sure that all the given digits in an equation like "$\pi=3.14159\dots$" are truly correct.

Ian
  • 101,645
  • I think that we can not "perfectly" define "correct up to $n$ decimal places" by reference to error, because there is always the counter-example that 1.000...0 estimates 0.999...9 correct up to $n$ decimal places, for a sufficiently long tail. – AgentSmith Feb 18 '20 at 15:08
2

It means rounding to $n$ decimal places.

Regarding $\pi$, note that $\pi = 3.14159\dots$. Therefore you round the third decimal place to $2$ instead of $1$.

molarmass
  • 2,014
  • 13
  • 16