0

I am struggling to find the correct implementation for rounding to the nearest multiple. I thought the simple arithmetic of [number/multiple]*multiple would give me my solution but I am running into instances where this is false. I am trying to mimic Excel's MROUND function.

Given the number 422.3710249 and the multiple .00125, I receive a number that is off by a certain level of precision... see javascript example to get the number.

console.log(Math.round(422.3710249/.00125)*.00125) = 422.37125000000003

Where my solution faulty? I expect this: 422.37125

S1r-Lanzelot
  • 133

1 Answers1

2

Remember that floating point numbers are not able to represent any real number. You are getting the right result using floating point arithmetic. The closest number to $422.37125$ the computer can represent using floats is $422.37125000000003$.

jjagmath
  • 18,214
  • It's more complicated than that. JavaScript uses IEEE 754 64-bit double precision throughout, and the two such numbers closest to $422.37125$ are $422.3712500000000318323\ldots$ and $422.3712499999999749888\ldots$ So you can see that the closest number to $422.37125$ should actually be displayed as $422.37124999999997$, not $422.37125000000003$. So I'm a bit puzzled by that, especially as the closest 64-bit representation of $0.00125$ is less than $0.00125$, so I would expect the result of the multiplication to be too small if anything. – TonyK Jul 29 '21 at 19:00