Varying results of package checks due to random seed
On 03/22/2014 01:32 PM, Radford Neal wrote:
From: Philippe GROSJEAN <Philippe.GROSJEAN at umons.ac.be> ... for latest CRAN version, we have successfully installed 4999 packages among the 5321 CRAN package on our platform. ... It is strange that a large portion of R CMD check errors on CRAN occur and disappear *without any version update* of a package or any of its direct or indirect dependencies! That is, a fraction of errors or warnings seem to appear and disappear without any code update.
Some of these are likely the result of packages running tests using random number generation without setting the random numbers seed, in which case the seed is set based on the current time and process id, with an obvious possibility of results varying from run to run. In the current development version of pqR (in branch 19-mods, found at https://github.com/radfordneal/pqR/tree/19-mods), I have implemented a change so that if the R_SEED environment variable is set, the random seed is initialized to its value, rather than from the time and process id. This was motivated by exactly this problem - I can now just set R_SEED to something before running all the package checks.
Beware, if you are serious about reproducing things, that you really need to save information about the uniform and other generators you use, such as the normal generator. The defaults do not change often, but have in the past, and could in the future if something better comes along. There are some small utilities and examples in the package setRNG which can help. Also remember that you need to beware of a side effect of the environment variable approach. It is great for reproducing things, as you would want to do in package tests, but be careful how you use it in functions as it may mess up the randomness if you always set the seed to the same starting value. Paul
Radford Neal
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel