1

Find the relative error when storing six digits in the mantissa using rounding of $\cos^2x-\sin^2x$ where $x=0.7854$

So $f(0.7854)=-3.673205105\cdot 10^{-6}$

And $fl(0.7854)=(0.7071)^2-(0.7071)^2=0$

So the relative error is $$\frac{\mid -3.673205105\cdot 10^{-6}-0\mid}{\mid -3.673205105\cdot 10^{-6}\mid}=100\%$$

But the answer in the book is $36%$ where did I get it wrong?

gbox
  • 12,867

1 Answers1

2

You need to round all intermediate values to 6 mantissa digits, for some reason you only used 4:

For the exact result, as you said

cos(2*0.7854)
 = -3.6732051033465739e-06

For the first step, the trigonometric values are 

(cos(0.7854), sin(0.7854))
 = (0.70710548251123628, 0.70710807985947355)

Now round to 6 digits after the decimal dot and square

(0.707105**2, 0.707108**2)
 = (0.49999748102499997, 0.500001723664)

and compute the difference of the squares, rounded again to 6 digits

0.499997-0.500002
 = -4.999999999921734e-06

The relative error is thus

-4.999999999921734e-06/-3.6732051033465739e-06-1
 = 0.36120904203425686

that is, the announced $36\%$.

Lutz Lehmann
  • 126,666
  • So we do not count the number before the radix as a significant number? So $0.707105=7.071054e-1$ is $6$ significant numbers? – gbox Feb 01 '18 at 10:44
  • 1
    No, that is why in the task they speak of "mantissa digits", as you used with scientific notation, that is, the leading zeros do not count. – Lutz Lehmann Feb 01 '18 at 10:48