0

Considering precision to the tenths place, where x is a non-zero integer, and the value has a tenths value of zero, Is there a precision difference between x and x.0 ?

Is one more accurate or mathematically correct?

Is this the same case for 0 vs 0.0?

Don
  • 103
  • 5

1 Answers1

1

Yes, there is a difference. If it is not an exact value, $x$ has a tolerance of $\pm 0.5$ while $x.0$ has a tolerance of $\pm 0.05$. Either is correct if it properly represents your state of knowledge of the value. $x.0$ promises more accuracy, but it may be an empty promise. Certainly $x.0$ should convince you the author has thought about it, while $x$ may mean the author hasn't really thought about the precision with which $x$ is known, or it could mean that $x$ is an exact value. Yes, I would say it is the same for $0$ vs $0.0$.

Ross Millikan
  • 374,822
  • I should have clarified. I have an array of values to the tenths place where zero is often the tenths value. Im curious about the general precision equality of leaving off the zero decimal and additionally if 0, 0.0 is any different. – Don May 23 '17 at 01:45
  • 1
    If you have a specific case, you should look into what the source of that data thinks. I believe my answer is the mainstream one, but some sources may do it differently. If all the values have a tenths digit, that should indicate that every value is $\pm 0.05$ Maybe they were all measured with some process that has that uncertainty. – Ross Millikan May 23 '17 at 02:42
  • In proper usage, they are not equivalent, as I said above. The claimed accuracy is different for the two cases. Do you still have a question? This seems the same as what was in the original post. – Ross Millikan May 24 '17 at 03:19