Skip to content

trunc/floor a number -- strange bahaviour

3 messages · Žroutík, Duncan Murdoch

#
I'm glad you are happy with your solution.  Just one inline comment 
below, expanding on my private reply to you:
?rout?k wrote:
No, that's not how standard floating point representations work.  12.34 is represented as a finite precision binary number between 1 and 2 (starting out as 1.10001 in binary notation) multiplied by 2^3.  The problem is that there is no way to represent it exactly in this format.  Only fractions with a power of 2 in the denominator can be represented that way, and 12.34 is not one of those.  It's 1234/100, and 100 is not a power of 2.

Duncan Murdoch