Skip to content
Prev 55901 / 63421 Next
Original Raw

Bias in R's random integers?

Ralf Stubner
On 9/21/18 6:38 PM, Tierney, Luke wrote:
There is an issue, but it is in vseq.c.

The plot I found striking was this:

http://people.redhat.com/sgrubb/files/r-random.jpg

It shows a scatter plot that is bounded to some rectangle where the
upper right and lower left corner are empty. Roughly speaking, X and Y
correspond to *consecutive differences* between random draws. It is
obvious that differences between random draws are bounded by the range
of the RNG, and that there cannot be two *differences in a row* that are
close to the maximum (or minimum). Hence the expected shape for such a
scatter plot is a rectangle with two corners being forbidden.

Within the allowed region, there should be no structure what so ever
(given enough draws). And that was striking about the above picture: It
showed clear vertical bands which should not be there. MT does fail some
statistical tests, but it cannot be brought down that easily.

Interestingly, I first used Dirk's C++ function for convenience, and
that did *not* show these bands. But when I compiled vseq.c I could
reproduce this. To cut this short: There is an error in vseq.c when the
numbers are read in:

    tmp = strtoul(buf, NULL, 16);

The third argument to strtoul is the base in which the numbers should be
interpreted. However, R has written numbers with base 10. Those can be
interpreted as base 16, but they will mean something different. Once one
changes the above line to

    tmp = strtoul(buf, NULL, 10);

the bands do disappear.

cheerio
ralf

Thread (35 messages)

Carl Boettiger Bias in R's random integers? Sep 18 Duncan Murdoch Bias in R's random integers? Sep 19 Iñaki Ucar Bias in R's random integers? Sep 19 David Hugh-Jones Bias in R's random integers? Sep 19 Ben Bolker Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Philip B. Stark Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 David Hugh-Jones Bias in R's random integers? Sep 19 Duncan Murdoch Bias in R's random integers? Sep 19 Ben Bolker Bias in R's random integers? Sep 19 Carl Boettiger Bias in R's random integers? Sep 19 Ralf Stubner Bias in R's random integers? Sep 20 Duncan Murdoch Bias in R's random integers? Sep 20 Paul Gilbert Bias in R's random integers? Sep 20 Gabriel Becker Bias in R's random integers? Sep 20 Hervé Pagès Bias in R's random integers? Sep 20 Steve Grubb Bias in R's random integers? Sep 20 Philip B. Stark Bias in R's random integers? Sep 20 Ralf Stubner Bias in R's random integers? Sep 21 Steve Grubb Bias in R's random integers? Sep 21 Dirk Eddelbuettel Bias in R's random integers? Sep 21 Dirk Eddelbuettel Bias in R's random integers? Sep 21 Luke Tierney Bias in R's random integers? Sep 21 Ralf Stubner Bias in R's random integers? Sep 21 Steve Grubb Bias in R's random integers? Sep 21 Steve Grubb Bias in R's random integers? Sep 21 Ralf Stubner Bias in R's random integers? Sep 27