G'day all,
I have daily scripts running to install the patched version of the
current R version and the development version of R on my linux box
(Ubuntu 18.04.2 LTS).
The last development version that was successfully compiled and
installed was "R Under development (unstable) (2019-02-25 r76159)" on
26 February. Since then the script always fails as a regression test
seems to fail. Specifically, in the tests/ subdirectory of my build
directory I have a file reg-tests-1d.Rout.fail which ends with:
+ all.equal(est$ar[1,,], diag(0.8, 2), tol = 0.08)# seen 0.0038
+ all.equal(est[1:6], es.[1:6], tol = 5e-3)
+ all.equal(estd$x.mean, es.d$x.mean, tol = 0.01) # seen 0.0023
+ all.equal(estd[c(1:3,5:6)],
+ es.d[c(1:3,5:6)], tol = 1e-3)## seen {1,3,8}e-4
+ all.equal(lapply(estd[1:6],unname),
+ lapply(est [1:6],unname), tol = 2e-12)# almost identical
+ all.equal(lapply(es.d[1:6],unname),
+ lapply(es. [1:6],unname), tol = 2e-12)
+ })
Error: lapply(es.d[1:6], unname) and lapply(es.[1:6], unname) are not equal:
Component "aic": Mean relative difference: 3.297178e-12
Execution halted
Would it be possible to make this tolerance more lenient? In case it
matters, I am configuring R to be compiled using Openblas and this test
fails for the 64 bit installation, the 32 bit installation seems to
pass all tests.
Happy to provide any more information/context that might be needed.
Cheers,
Berwin
Berwin A Turlach
on Tue, 5 Mar 2019 16:54:09 +0800 writes:
> G'day all,
> I have daily scripts running to install the patched version of the
> current R version and the development version of R on my linux box
> (Ubuntu 18.04.2 LTS).
> The last development version that was successfully compiled and
> installed was "R Under development (unstable) (2019-02-25 r76159)" on
> 26 February. Since then the script always fails as a regression test
> seems to fail. Specifically, in the tests/ subdirectory of my build
> directory I have a file reg-tests-1d.Rout.fail which ends with:
>> ## checking ar.yw.default() multivariate case
>> estd <- ar(unclass(y) , aic = FALSE, order.max = 2) ## Estimate VAR(2)
>> es.d <- ar(unclass(y.), aic = FALSE, order.max = 2, na.action=na.pass)
>> stopifnot(exprs = {
> + all.equal(est$ar[1,,], diag(0.8, 2), tol = 0.08)# seen 0.0038
> + all.equal(est[1:6], es.[1:6], tol = 5e-3)
> + all.equal(estd$x.mean, es.d$x.mean, tol = 0.01) # seen 0.0023
> + all.equal(estd[c(1:3,5:6)],
> + es.d[c(1:3,5:6)], tol = 1e-3)## seen {1,3,8}e-4
> + all.equal(lapply(estd[1:6],unname),
> + lapply(est [1:6],unname), tol = 2e-12)# almost identical
> + all.equal(lapply(es.d[1:6],unname),
> + lapply(es. [1:6],unname), tol = 2e-12)
> + })
> Error: lapply(es.d[1:6], unname) and lapply(es.[1:6], unname) are not equal:
> Component "aic": Mean relative difference: 3.297178e-12
> Execution halted
> Would it be possible to make this tolerance more lenient? In case it
> matters, I am configuring R to be compiled using Openblas and this test
> fails for the 64 bit installation, the 32 bit installation seems to
> pass all tests.
Interesting. But we had another report (on this mailing list!) by
Er. Ho., also using OpenBLAS, but then in combination with Cygwin.
After Cygwin related problems were solved, on March 1 he said
I have an error at reg-1d. It stops the process. However, the mean difference
(as per the file) is 2.0e-12.
and when I asked him to be more precise.. he wasn't but did
"solve" the problem by changing the test file.
Now we know from you Berwin (thank you for providing the details!)
what it has been.
==> I could investigate and -- lo and behold! -- the solution
is probably the fact that the RNGkind(sample.kind = *) was
introduced *and* the default was to differ from previous
versions of R.
> Happy to provide any more information/context that might be needed.
Can you please try adding
suppressWarnings(RNGversion("3.5.0"))
e.g. at the very beginning of the tests/reg-tests-1d.R
file or just a few lines above the code you show above,
replace in line 1470
set.seed(42)
by
set.seed(42); suppressWarnings(RNGversion("3.5.0"))
It will use a different missingness pattern for that
multivariate AR example, i.e., different data.
Best, Martin
G'day all,
I have daily scripts running to install the patched version of the
current R version and the development version of R on my linux box
(Ubuntu 18.04.2 LTS).
The last development version that was successfully compiled and
installed was "R Under development (unstable) (2019-02-25 r76159)" on
26 February. Since then the script always fails as a regression test
seems to fail. Specifically, in the tests/ subdirectory of my build
directory I have a file reg-tests-1d.Rout.fail which ends with:
+ all.equal(est$ar[1,,], diag(0.8, 2), tol = 0.08)# seen 0.0038
+ all.equal(est[1:6], es.[1:6], tol = 5e-3)
+ all.equal(estd$x.mean, es.d$x.mean, tol = 0.01) # seen 0.0023
+ all.equal(estd[c(1:3,5:6)],
+ es.d[c(1:3,5:6)], tol = 1e-3)## seen {1,3,8}e-4
+ all.equal(lapply(estd[1:6],unname),
+ lapply(est [1:6],unname), tol = 2e-12)# almost identical
+ all.equal(lapply(es.d[1:6],unname),
+ lapply(es. [1:6],unname), tol = 2e-12)
+ })
Error: lapply(es.d[1:6], unname) and lapply(es.[1:6], unname) are not equal:
Component "aic": Mean relative difference: 3.297178e-12
Execution halted
Would it be possible to make this tolerance more lenient? In case it
matters, I am configuring R to be compiled using Openblas and this test
fails for the 64 bit installation, the 32 bit installation seems to
pass all tests.
Happy to provide any more information/context that might be needed.
The version of OpenBLAS is always helpful (a couple of bugs were fixed
recently that impacted the checks in R packages).
I have a x86_64 Linux build with OpenBLAS 0.3.5 that I use for the
'Additional issues' in package checking, and that passes its checks (and
has ca weekly for years, even with the buggy versions of OpenBLAS).
Some aspects of the RNG were changed in r76160: to isolate change that
you can (with current R-devel) use
setenv _R_RNG_VERSION_ 3.5.0
and re-check.
Brian D. Ripley, ripley at stats.ox.ac.uk
Emeritus Professor of Applied Statistics, University of Oxford
Martin Maechler
on Tue, 5 Mar 2019 12:45:36 +0100 writes:
Berwin A Turlach
on Tue, 5 Mar 2019 16:54:09 +0800 writes:
>> G'day all,
>> I have daily scripts running to install the patched version of the
>> current R version and the development version of R on my linux box
>> (Ubuntu 18.04.2 LTS).
>> The last development version that was successfully compiled and
>> installed was "R Under development (unstable) (2019-02-25 r76159)" on
>> 26 February. Since then the script always fails as a regression test
>> seems to fail. Specifically, in the tests/ subdirectory of my build
>> directory I have a file reg-tests-1d.Rout.fail which ends with:
>>> ## checking ar.yw.default() multivariate case
>>> estd <- ar(unclass(y) , aic = FALSE, order.max = 2) ## Estimate VAR(2)
>>> es.d <- ar(unclass(y.), aic = FALSE, order.max = 2, na.action=na.pass)
>>> stopifnot(exprs = {
>> + all.equal(est$ar[1,,], diag(0.8, 2), tol = 0.08)# seen 0.0038
>> + all.equal(est[1:6], es.[1:6], tol = 5e-3)
>> + all.equal(estd$x.mean, es.d$x.mean, tol = 0.01) # seen 0.0023
>> + all.equal(estd[c(1:3,5:6)],
>> + es.d[c(1:3,5:6)], tol = 1e-3)## seen {1,3,8}e-4
>> + all.equal(lapply(estd[1:6],unname),
>> + lapply(est [1:6],unname), tol = 2e-12)# almost identical
>> + all.equal(lapply(es.d[1:6],unname),
>> + lapply(es. [1:6],unname), tol = 2e-12)
>> + })
>> Error: lapply(es.d[1:6], unname) and lapply(es.[1:6], unname) are not equal:
>> Component "aic": Mean relative difference: 3.297178e-12
>> Execution halted
>> Would it be possible to make this tolerance more lenient? In case it
>> matters, I am configuring R to be compiled using Openblas and this test
>> fails for the 64 bit installation, the 32 bit installation seems to
>> pass all tests.
> Interesting. But we had another report (on this mailing list!) by
> Er. Ho., also using OpenBLAS, but then in combination with Cygwin.
> After Cygwin related problems were solved, on March 1 he said
>> I have an error at reg-1d. It stops the process. However, the mean difference
>> (as per the file) is 2.0e-12.
> and when I asked him to be more precise.. he wasn't but did
> "solve" the problem by changing the test file.
> Now we know from you Berwin (thank you for providing the details!)
> what it has been.
> ==> I could investigate and -- lo and behold! -- the solution
> is probably the fact that the RNGkind(sample.kind = *) was
> introduced *and* the default was to differ from previous
> versions of R.
>> Happy to provide any more information/context that might be needed.
> Can you please try adding
> suppressWarnings(RNGversion("3.5.0"))
> e.g. at the very beginning of the tests/reg-tests-1d.R
> file or just a few lines above the code you show above,
> replace in line 1470
> set.seed(42)
> by
> set.seed(42); suppressWarnings(RNGversion("3.5.0"))
oops! That was a thinko (enticed by my "esthetics"): Of
course you must *first* set the RNG version and *then* the seed,
so *only* this is correct
suppressWarnings(RNGversion("3.5.0")); set.seed(42)
where I really would go for
suppressWarnings(RNGversion("3.5.0")) # back compatibility for now
set.seed(42)
Note that Brian's advice with an environment variable currently
works "more thoroughly", but it has been labeled "temporary" as it
is not functional / language-explicit [i.e. not visible from
your own R code] ...
and you really should only use it when checking R, *not* otherwise.
Martin