[Bioc-devel] Redirect workers output to STDERR, how to do so with current BiocParallel::SnowParam()?

Hi,

This might be a BioC support website question, but maybe it's a bug.

In previous versions of BiocParallel, you could specify where to
direct the output from the workers by using the 'outfile' argument.
For example, SnowParam(outfile = Sys.getenv('SGE_STDERR_PATH')). I'm
not finding how to do so with the current version (1.3.12).

I understand that SnowParam() has a ... argument that gets passed to
snow::makeCluster(), according to the SnowParam() docs. I also see
that when log = TRUE, a script is used instead of snow. But log =
FALSE by default.

If I use snow, the 'outfile' argument does work as shown below:

## Nothing gets printed by default

cl <- makeCluster(2, type = 'SOCK')
y <- clusterApply(cl, 1:2, print)

## Use outfile now, print works

cl <- makeCluster(2, type = 'SOCK', outfile = NULL)

starting worker for localhost:11671
starting worker for localhost:11671

y <- clusterApply(cl, 1:2, print)

Type: EXEC
Type: EXEC
[1] 1
[1] 2

packageVersion('snow')

[1] ?0.3.13?


However, I can't use 'outfile' with SnowParam:

## SerialParam works

x <- bplapply(1:2, print, BPPARAM = SerialParam())

[1] 1
[1] 2

## No printing by default with SnowParam, as expected

x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1))

## Can't pass 'outfile' argument

x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, outfile = NULL))

Error in bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, outfile
= NULL)) :
   error in evaluating the argument 'BPPARAM' in selecting a method for
function 'bplapply': Error in envRefSetField(.Object, field, classDef,
selfEnv, elements[[field]]) :
   ?outfile? is not a field in class ?SnowParam?


Digging at the code, I see that in SnowParam() the ... argument is
saved in .clusterargs

args <- c(list(spec = workers, type = type), list(...))
     .clusterargs <- lapply(args, force)

However, ... is still passed to .SnowParam(), and .SnowParam() fields are:

fields=list(
         cluster="cluster",
         .clusterargs="list",
         .controlled="logical",
         log="logical",
         threshold="ANY",
         logdir="character",
         resultdir="character")


So, I'm wondering if ... should not be passed to .SnowParam().



## Trying to pass outfile to .clusterargs directly doesn't work
## Actually, I'm surprised it didn't crash .SnowParam()

x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, .clusterargs=list(outfile = NULL)))

Full log: https://gist.github.com/9ac957c6cad1c07f4ea4


Thanks,
Leo