Skip to content
Prev 55890 / 63421 Next
Original Raw

Bias in R's random integers?

Paul Gilbert
On 09/19/2018 10:03 AM, Ben Bolker wrote:
...
I think improvements in the RNG is a situation where backward 
compatibility is not really going to be lost, because people can specify 
the old generator, they just will not get it by default. My opinion is 
that the default needs to generally be the best option available because 
too many people will be expecting that, or not know better, in which 
case that is what they should get.

There are only two small problems that occur to me:

1/ Researchers that want to have reproducible results (all I hope) need 
to be aware the change has happened. In theory they should have recorded 
the RNG they were using, along with the seed (and, BTW, the number of 
nodes if they generate with a parallel generator). If they have not done 
that then they can figure out the RNG from knowing what version of R 
they used. If they haven't recorded that then they can figure it out by 
some experimentation and knowing roughly when they did the research. If 
none of this works then the research probably should be lost.

As an exercise, researchers might also want to experiment with whether 
the new default qualitatively changes their results. That might lead to 
publishable research, so no one should complain.

2/ Package maintainers that have used the default RNG to generate tests 
may need to change their tests to specify the old generator, or modify 
results used for comparisons in the tests. Since package testing is 
usually for code checking rather than statistical results, not using the 
best available generator is not usually an issue.

Most of my own package testing already specifies the generator, lots 
uses "buggy Kinderman-Ramage" because tests were set up a long time ago. 
I will have to change package setRNG which warns when the default 
generator changes. (This warning is intentional because I was bitten 
badly by a small change in the S generator circa 1990.)
I think there may have been a change in R 0.99 too. At least my notes 
suggest that the code I changed for  R 1.7.0 had worked with the default 
generator from R 0.99 to 1.6.2.

I don't recall the protocol, I think it just happened and was announced 
in the NEWS. (Has this protocol changed?) The ramification for me was 
that I had to go through all of my packages' testing and change the name 
of the explicitly specified RNG to "buggy Kinderman-Ramage".

Perhaps there does need to be a protocol for testing before release. 
When my package setRNG fails then many of my other packages will also 
fail because they depend on it. This is a simple fix but reverse 
dependencies may make it look like lots of things are broken.

Paul Gilbert

Thread (35 messages)

Carl Boettiger Bias in R's random integers? Sep 18 Duncan Murdoch Bias in R's random integers? Sep 19 Iñaki Ucar Bias in R's random integers? Sep 19 David Hugh-Jones Bias in R's random integers? Sep 19 Ben Bolker Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 David Hugh-Jones Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Ben Bolker Bias in R's random integers? Sep 19 Carl Boettiger Bias in R's random integers? Sep 19 Ralf Stubner Bias in R's random integers? Sep 20 Duncan Murdoch Bias in R's random integers? Sep 20 Paul Gilbert Bias in R's random integers? Sep 20 Gabriel Becker Bias in R's random integers? Sep 20 Hervé Pagès Bias in R's random integers? Sep 20 Steve Grubb Bias in R's random integers? Sep 20 Philip B. Stark Bias in R's random integers? Sep 20 Ralf Stubner Bias in R's random integers? Sep 21 Steve Grubb Bias in R's random integers? Sep 21 Dirk Eddelbuettel Bias in R's random integers? Sep 21 Dirk Eddelbuettel Bias in R's random integers? Sep 21 Luke Tierney Bias in R's random integers? Sep 21 Ralf Stubner Bias in R's random integers? Sep 21 Steve Grubb Bias in R's random integers? Sep 21 Steve Grubb Bias in R's random integers? Sep 21 Ralf Stubner Bias in R's random integers? Sep 27