Questions tagged [catastrophic-cancellation]

For questions about catastrophic cancellation, the devastating loss of precision when small numbers are computed from large numbers, which themselves are subject to roundoff error.

Catastrophic cancellation is effect which appears in numerical computations, when in subtracting two numbers or some other operation the final result is smaller than rounding error.

21 questions
1
vote
1 answer

Loss of Significance in Experiment

I am quite familiar with the problem associated with loss of significance in numerical analysis. This problem seems to be an even bigger problem in the science lab, where subtracting one physical measurement from another produces potentially…