On Mon, Feb 6, 2017 at 12:41 AM, G?bor Cs?rdi <csardi.gabor at gmail.com>
wrote:
I don't think it is specific for testthat. R CMD check sets R_TESTS when
runs the tests, so the separate R process it starts can have some special
startup options.
The problem happens if you start another R process from your R test, and
then R_TESTS confuses this process. (I am not sure how exactly.)
I wasn't aware of R_TESTS. Looking at the R source code, if set, will
cause the base package to source that file _before_ and .Rprofile
scripts; it basically calls source(Sys.getenv("R_TESTS")).
Next, R CMD check runs each of the test script using
R_TESTS=startup.Rs, where startup.Rs:
file.copy(file.path(R.home("share"), "R", "tests-startup.R"),
"startup.Rs")
if (use_gct) cat("gctorture(TRUE)" , file = "startup.Rs", append =
TRUE)
Looking at the tests-startup.R file that comes with my R 3.3.2 on
Ubuntu 16.04, it seems pretty harmless:
## A custom startup file for tests
## Run as if a system Rprofile, so no packages, no assignments
options(useFancyQuotes = FALSE)
So, to me it's not clear how this could make a difference in Patrick
case. By disabling this, i.e. Sys.setenv(R_TESTS=""), I don't see
how it affects running parallel+foreach+doParallel. Maybe because one
of those packages are relying on fancy quotes in some protocol,
passing command-line arguments or ... something. I would be curious
to see what file.path(R.home("share"), "R", "tests-startup.R")
contains on the system where the problem occurs.
I wonder if those are mostly there because of cut'n'paste behavior.
Most of these use testthat, but not all of them.
My interest in this issue is because I haven't yet experienced this
myself and I run lots and lots of package tests in future that
utilizes the parallel package. In doFuture I do similar tests, which
is on top of the foreach package. I don't use testthat and I also
don't use doParallel in my testing.
/Henrik
Gabor
On Mon, Feb 6, 2017 at 2:17 AM, Henrik Bengtsson
<henrik.bengtsson at gmail.com> wrote:
In case someone else bumps into this later and finds this thread; can
you confirm that this was a problem specific to using the testthat
package for running the tests?
/Henrik
On Sun, Feb 5, 2017 at 11:28 AM, Patrick Schratz
<patrick.schratz at gmail.com> wrote:
Thanks for the hint, Hendrik!
However, this change did not make a difference :/
I tried to use all cluster closing functions I came across but tests
are
still running infinite..
cl <- makeCluster(par.args$par.units, outfile = out.progress)
registerDoParallel(cl)
foreach()
parallel::stopCluster(cl)
doParallel::registerDoSEQ()
doParallel::stopImplicitCluster()
2017-02-05 15:04 GMT+01:00 Henrik Bengtsson
<henrik.bengtsson at gmail.com>:
Use
registerDoParallel(cl)
The number of parallel workers is already contained in the 'cl'
object,
so don't specify 'cores'! (If you do that, I suspect you create yet
another
cluster (a multicore one) which is used but never closed)
registerDoParallel() should ideally give an error in your case.
BCC:ed.
Henrik
On Feb 5, 2017 03:56, "Patrick Schratz" <patrick.schratz at gmail.com>
wrote:
Dear Uwe,
thanks for the hint. My cluster is closed after the `foreach`call
using
`stopCluster()`.
Before, I?ll do the following to init the cluster:
*cl <- makeCluster(par.args$par.units, outfile = out.progress)*
*registerDoParallel(cl, cores = par.args$par.units)*
*foreach()*
*stopCluster(cl)*
Do you know of any other package which is using foreach in
combination
with
tests and is hosted on Github? So I could compare settings.
Best, Patrick
2017-02-02 0:01 GMT+01:00 Uwe Ligges
<ligges at statistik.tu-dortmund.de>:
Check whether the parallel cluster is closed. Can it be that the
cluster
is still open and the check process waits for them to complete?
Best,
Uwe Ligges
On 31.01.2017 13:45, Patrick Schratz wrote:
Hello,
when running R CMD check / devtools::check, section "running
tests..." is
not finishing (40 min+).
*Checking tests only works:*
*==> Sourcing R files in 'tests' directory*
*testthat results
============================================================
*OK: 7 SKIPPED: 0 FAILED: 0*
*Tests complete*
As well as running tests line-by-line.
How can I debug my tests to discover the problem during R CMD
check?
*Tests are using parallelization (foreach + doParallel)*
Best, Patrick
[[alternative HTML version deleted]]