[Bioc-devel] parallel processing in R?

Hi Bhakti,

On Mon, Dec 11, 2017 at 12:19 PM, Dwivedi, Bhakti <bhakti.dwivedi at emory.edu>
wrote:

Is there a way to parallelize ConsensusClusterPlus package?
https://bioconductor.org/packages/release/bioc/html/
ConsensusClusterPlus.html
We are developing a R shiny tool that performs consensus clustering in
addition to other genome-wide analyses. The consensus clustering step is
taking the longest.
Can I do parallel processing in R/R shiny? What parallel package (if any)
I can implement in the R code to do parallel computing?

The canonical package for parallel computing in Bioconductor is
BiocParallel:
https://bioconductor.org/packages/release/bioc/html/BiocParallel.html
One can choose what backend to use for parallelism.  You can switch from
using SerialParam for cheap and cheerful lapply-like functionality to the
default MultiCoreParam and SnowParam which nicely logs useful things like
memory usage.

It does not look like ConsensusClusterPlus is importing any parallel
package of it's own that you need to fight against, so best case scenario
is you look at the function of interest you want to run many times and run
that function with BiocParallel's bplapply.  Or if there are multiple
levels of parallelism like internal and external looping then you might
have to dive into ConsensusClusterPlus and inject bplapply statements
ideally allowing some bpparam() argument passing for the inner and outer
loops.

Bhakti

Pariksheet