On Tue, Feb 7, 2017 at 8:16 PM, Henrik Bengtsson <
henrik.bengtsson at gmail.com> wrote:
[...]
So, to me it's not clear how this could make a difference in Patrick
case. By disabling this, i.e. Sys.setenv(R_TESTS=""), I don't see
how it affects running parallel+foreach+doParallel. Maybe because one
of those packages are relying on fancy quotes in some protocol,
passing command-line arguments or ... something. I would be curious
to see what file.path(R.home("share"), "R", "tests-startup.R")
contains on the system where the problem occurs.
I investigated a bit, and the problem (or at least one problem) is that
the R subprocess will try
to source() startup.Rs from the working directory, and that file might not
be there.
For testthat, the tests run in tests/testthat, and it is typically not
there.
For other regular tests, RUnit, etc. AFAIK the tests run in tests/ and
startup.Rs is
there. Unless the tests change the working directory, in which case it is
not.
So another workaround is to put an empty startup.Rs file there.
Maybe there are other problems as well, but just putting an empty
startup.Rs file at the appropriate
location fixed all cases I tried.
The parallel case above in this thread is a bit unusual I think, because
usually there
is no freeze, the R subprocess just aborts, and this typically causes a
testthat (or other) test
failure.
Nevertheless, putting an empty startup.Rs file in tests/testthat fixes the
testParallel
package as well.
G
I wonder if those are mostly there because of cut'n'paste behavior.
Most of these use testthat, but not all of them.
My interest in this issue is because I haven't yet experienced this
myself and I run lots and lots of package tests in future that
utilizes the parallel package. In doFuture I do similar tests, which
is on top of the foreach package. I don't use testthat and I also
don't use doParallel in my testing.
/Henrik
Gabor
On Mon, Feb 6, 2017 at 2:17 AM, Henrik Bengtsson
<henrik.bengtsson at gmail.com> wrote:
In case someone else bumps into this later and finds this thread; can
you confirm that this was a problem specific to using the testthat
package for running the tests?
/Henrik
On Sun, Feb 5, 2017 at 11:28 AM, Patrick Schratz
<patrick.schratz at gmail.com> wrote:
patrick.schratz at gmail.com>:
Thanks for the hint, Hendrik!
However, this change did not make a difference :/
I tried to use all cluster closing functions I came across but tests
are
still running infinite..
cl <- makeCluster(par.args$par.units, outfile = out.progress)
registerDoParallel(cl)
foreach()
parallel::stopCluster(cl)
doParallel::registerDoSEQ()
doParallel::stopImplicitCluster()
2017-02-05 15:04 GMT+01:00 Henrik Bengtsson
<henrik.bengtsson at gmail.com>:
Use
registerDoParallel(cl)
The number of parallel workers is already contained in the 'cl'
object,
so don't specify 'cores'! (If you do that, I suspect you create
another
cluster (a multicore one) which is used but never closed)
registerDoParallel() should ideally give an error in your case.
BCC:ed.
Henrik
On Feb 5, 2017 03:56, "Patrick Schratz" <patrick.schratz at gmail.com
Dear Uwe,
thanks for the hint. My cluster is closed after the `foreach`call
using
`stopCluster()`.
Before, I?ll do the following to init the cluster:
*cl <- makeCluster(par.args$par.units, outfile = out.progress)*
*registerDoParallel(cl, cores = par.args$par.units)*
*foreach()*
*stopCluster(cl)*
Do you know of any other package which is using foreach in
combination
with
tests and is hosted on Github? So I could compare settings.
Best, Patrick
2017-02-02 0:01 GMT+01:00 Uwe Ligges
<ligges at statistik.tu-dortmund.de>:
Check whether the parallel cluster is closed. Can it be that the
cluster
is still open and the check process waits for them to complete?
Best,
Uwe Ligges
On 31.01.2017 13:45, Patrick Schratz wrote:
Hello,
when running R CMD check / devtools::check, section "running
tests..." is
not finishing (40 min+).
*Checking tests only works:*
*==> Sourcing R files in 'tests' directory*
*testthat results
============================================================
*OK: 7 SKIPPED: 0 FAILED: 0*
*Tests complete*
As well as running tests line-by-line.
How can I debug my tests to discover the problem during R CMD
check?
*Tests are using parallelization (foreach + doParallel)*
Best, Patrick
[[alternative HTML version deleted]]