Precision in R
On 1/15/2009 1:32 PM, David Winsemius wrote:
On Jan 15, 2009, at 12:25 PM, Charles C. Berry wrote:
This is what I get on windows XP:
tcp1-tcp2
[,1] [,2] [,3] [1,] -2.861023e-06 -4.768372e-07 -4.768372e-07 [2,] -4.768372e-07 -3.814697e-06 2.622604e-06 [3,] -4.768372e-07 2.622604e-06 -5.960464e-08
but on my Gentoo Linux Intel Core 2 Duo:
print(tcp1-tcp2,digits=20)
[,1] [,2] [,3]
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
So, it would seem to be that the floating point calcs on my Windows box, David's Mac, and whatever system you are using are not as accurate as they might be.
For the record; on an Intel Mac (Leopard) with Urbanek's compilation of R 2.8.1
> print(tcp1-tcp2,digits=20)
[,1] [,2] [,3] [1,] 0 0 0 [2,] 0 0 0 [3,] 0 0 0
I haven't debugged these, but I would guess this is because we have the Windows libraries set to use extended precision (64 bit mantissas) on intermediate results, whereas once things get stored to RAM, they are rounded to double precision (53 bit mantissas). This has the benefit of giving a more accurate answer in many circumstances, but the disadvantage that the final results are more dependent on the order of calculations. I think the other platforms never do the full extended precision calculations, so their results are consistent (but probably less accurate sometimes). Duncan Murdoch