[Bioc-devel] C++ parallel computing

Incidentally, I was reflecting on this topic the other day and was 
wondering whether BiocParallel could have something like OpenMPParam() 
that sets the number of threads to some non-zero value via 
omp_set_num_threads(). This would provide a consistent framework through 
which users could control OpenMP behavior in suitably written functions.

One could even imagine having a composition design where a caller could 
assemble a BPPARAM object like:

bplapply(..., BPPARAM=OpenMPParam(SnowParam(5), 2))

which tells bplapply to spin up 5 workers where each worker is allowed 
to use up to 2 threads each. Implementation-wise, it would be a 
relatively simple matter of stuffing an extra set-up command into 
.composeTry; the nthread-setting code can be borrowed from ShortRead.

For context: I am planning on moving more parallelization in my packages 
into OpenMP to get around the overhead of the other backends. Forking is 
the only approach that is remotely fast enough, but the interaction of 
forks with the GC is too chaotic in memory-limited environments.

-A

[Bioc-devel] C++ parallel computing

Thread (6 messages)