Skip to content
Prev 58892 / 398502 Next

wrong answer for simple expressions

On Thu, 2004-11-11 at 12:24 -0500, Drew Hoysak wrote:
A lack of understanding as to how floating point numbers are represented
by computers under the IEEE 754 floating point standard. 

Hint: Take note of the following:
[1] 0.10000000000000000555
[1] 4.0999999999999996447
[1] 0.99999999999999955591
[1] -4.0999999999999996447


Read the last FAQ "Why is 0.1 not 0.1?" here:

http://grouper.ieee.org/groups/754/faq.html#binary-decimal

and read David Goldberg's article, "What Every Computer Scientist Should
Know about Floating-Point Arithmetic", which is available here:

http://grouper.ieee.org/groups/754/

in a Postscript file or here in an edited form in HTML:

http://docs.sun.com/source/806-3568/ncg_goldberg.html


HTH,

Marc Schwartz