Skip to content

number point under-flow

6 messages · Arne.Muller@aventis.com, Roger D. Peng, Bob Wheeler +2 more

#
Hello,

I've come across the following situation in R-1.8.1 (compile + running under
RedHat 7.1):
[1] 1
[1] -1.037310e-11

I'd expect the later to be 0 or some very small positive number. Is this a
number under-flow of the calculation? Do you think I'm safe if I just set the
result to 0 in these cases?

	kind regards,

	Arne
#
Did you compile with gcc-2.96?  I think there were some 
problems with the floating point arithmetic with that 
compiler (at least for the earlier versions released by Red 
Hat).

-roger
Arne.Muller at aventis.com wrote:
#
It's not the compiler. pghyper() in SuppDists does the same thing. Its 
just rounding error. Set the result to 0 if it bothers you.

pghyper(24,514,53, 5961, lower.tail=F)
[1] -3.325965e-12
Roger D. Peng wrote:

  
    
#
Roger D. Peng wrote:

            
Not (solely) a gcc-2.96 bug, since we get

   phyper(24, 514, 5961-514, 53, lower.tail=F)
[1] -3.325784e-12

on Windows compiled with gcc-3.3.1

Uwe Ligges
#
Try the latest patched version, as this has already been fixed.
[1] 1.084688e-13

    o   [l]choose() use a more accurate formula which also slightly
        improves p- and qhyper(); choose(n, k) now returns 0 instead
        of NaN for k < 0 or > n.
On Wed, 4 Feb 2004 Arne.Muller at aventis.com wrote:

            

  
    
#
Note R-patched and R-devel get more accurate answers.
On Wed, 4 Feb 2004, Uwe Ligges wrote: