Skip to content
Prev 58888 / 398502 Next

wrong answer for simple expressions

On Thu, 11 Nov 2004, Drew Hoysak wrote:

            
No. There is no reason why these expressions should yield zero values. 
Remember that computers work in base 2, and that 0.1 has an infinitely 
recurring binary expansion in base 2.  You should expect that 0.1 computed 
two different ways should differ in the last few bits. You have managed to 
get zero to 52 bits accuracy, which is not bad when you consider that the 
machine only works to 54 bits.

 	-thomas
Thomas Lumley			Assoc. Professor, Biostatistics
tlumley at u.washington.edu	University of Washington, Seattle