[Bioc-devel] BiocParallel-devel error

Hi Thomas,

Just wanted to let you know I saw this and am looking into it.

Valerie
Hi Martin, Micheal and Vincent,

If I run the following code, with the release version of BiocParallel then it
works (took me some time to actually realize that), but with the development
version I am getting an error shown after the test code below. If I run the
same test with BatchJobs from the devel branch alone then there is no problem.
Thus, it seems there is some change in the devel version of BiocParallel
causing this error? The torque.tmpl file I am using on our cluster is the
standard one from BatchJobs here:
https://github.com/tudo-r/BatchJobs/blob/master/examples/cfTorque/simple.tmpl

For my application, I could stick with BatchJobs, but it would be nicer if I
could get things to work with BiocParallel.

Thanks,

Thomas

###############
## Test Code ##
###############
FUN <- function(i) system("hostname", intern=TRUE)
library(BiocParallel); library(BatchJobs)
funs <- makeClusterFunctionsTorque("torque.tmpl")
param <- BatchJobsParam(4, resources=list(walltime="48:00:00", nodes="1:ppn=4", memory="4gb"), cluster.functions=funs)
register(param)
xx <- bplapply(1:4, FUN)

Error: 4 errors; first error:

For more information, use bplasterror(). To resume calculation, re-call the function and
set the argument 'BPRESUME' to TRUE or wrap the previous call in bpresume()

bplasterror()
Error in vapply(head(which(is.error), n.print), f, character(1L)) :
   values must be length 1,
  but FUN(X[[1]]) result is length 0

sessionInfo()
R Under development (unstable) (2014-05-05 r65530)
Platform: x86_64-unknown-linux-gnu (64-bit)

locale:
[1] C

attached base packages:
[1] stats     graphics  utils     datasets  grDevices methods   base

other attached packages:
[1] BatchJobs_1.3        BBmisc_1.7           BiocParallel_0.99.19

loaded via a namespace (and not attached):
  [1] BiocGenerics_0.11.4 DBI_0.3.0           RSQLite_0.11.4      brew_1.0-6          checkmate_1.4       codetools_0.2-9     digest_0.6.4        fail_1.2            foreach_1.4.2       iterators_1.0.7
[11] parallel_3.2.0      sendmailR_1.1-2     stringr_0.6.2       tools_3.2.0

_______________________________________________
Bioc-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Hi,

Martin and I looked into this a bit. It looks like a problem with 
handling an 'undefined error' returned from a worker (i.e., job did not 
run). When there is a problem executing the tmpl script no error message 
is sent back. The NULL is coerced to simpleError and becomes a problem 
downstream when the error processing is expecting messages of length > 0.

You can reproduce the error by putting a typo in the script. For example 
replace R with something bogus such as MYR in this line:

MYR CMD --no-save --no-restore "<%= rscript %>" /dev/stdout

You said the script worked with release but not devel. Is it possible 
there's a problem with how R devel is being called on the cluster?

Michel Lang (cc'd) implemented BatchJobs in BiocParallel. I'd like to 
get his opinion on how he wants to handle this type of error.
Michel, let me know if you need more details, I can send another example 
off-line.

Valerie
Hi Thomas,

Just wanted to let you know I saw this and am looking into it.

Valerie

On 09/20/2014 02:54 PM, Thomas Girke wrote:
Hi Martin, Micheal and Vincent,

If I run the following code, with the release version of BiocParallel
then it
works (took me some time to actually realize that), but with the
development
version I am getting an error shown after the test code below. If I
run the
same test with BatchJobs from the devel branch alone then there is no
problem.
Thus, it seems there is some change in the devel version of BiocParallel
causing this error? The torque.tmpl file I am using on our cluster is the
standard one from BatchJobs here:
https://github.com/tudo-r/BatchJobs/blob/master/examples/cfTorque/simple.tmpl

For my application, I could stick with BatchJobs, but it would be
nicer if I
could get things to work with BiocParallel.

Thanks,

Thomas

###############
## Test Code ##
###############
FUN <- function(i) system("hostname", intern=TRUE)
library(BiocParallel); library(BatchJobs)
funs <- makeClusterFunctionsTorque("torque.tmpl")
param <- BatchJobsParam(4, resources=list(walltime="48:00:00",
nodes="1:ppn=4", memory="4gb"), cluster.functions=funs)
register(param)
xx <- bplapply(1:4, FUN)

Error: 4 errors; first error:

For more information, use bplasterror(). To resume calculation,
re-call the function and
set the argument 'BPRESUME' to TRUE or wrap the previous call in
bpresume()

bplasterror()
Error in vapply(head(which(is.error), n.print), f, character(1L)) :
   values must be length 1,
  but FUN(X[[1]]) result is length 0

sessionInfo()
R Under development (unstable) (2014-05-05 r65530)
Platform: x86_64-unknown-linux-gnu (64-bit)

locale:
[1] C

attached base packages:
[1] stats     graphics  utils     datasets  grDevices methods   base

other attached packages:
[1] BatchJobs_1.3        BBmisc_1.7           BiocParallel_0.99.19

loaded via a namespace (and not attached):
  [1] BiocGenerics_0.11.4 DBI_0.3.0           RSQLite_0.11.4
brew_1.0-6          checkmate_1.4       codetools_0.2-9
digest_0.6.4        fail_1.2            foreach_1.4.2
iterators_1.0.7
[11] parallel_3.2.0      sendmailR_1.1-2     stringr_0.6.2
tools_3.2.0

_______________________________________________
Bioc-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

_______________________________________________
Bioc-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel