Skip to content
Prev 52643 / 63424 Next
Original Raw

[parallel-package] feature request: set default cluster type via environment variable

Christian Krause 
Dear all,

I?m working as an administrator of a High-Performance Computing (HPC) Cluster which runs on Linux. A lot of people are using R on this Linux cluster and, of course, the *parallel* package to speed up their computations.

It has been our collective experience, that using |makeForkCluster| yields an overall better experience /on Linux/ than the |makePSOCKcluster|, for whatever definition of better. Let me just summarize that it works smoother. I believe, other people working with *parallel* on Linux can share this experience

Also, we did really welcome the environment variable |MC_CORES|, to be able to specify (in job submit scripts) the amount of CPU cores a user has been granted, most importantly for /dynamic resource requests/ (see below for an example).

What we would also appreciate - and now we finally get to the feature request - is another environment variable to choose the used cluster, as in:

|export MC_CLUSTER_TYPE=FORK |

Do you think something like this could be implemented in future releases?


      Parallel R job submit script

This works with the Univa Grid Engine and should work with other * Grid Engine products:

|#!/bin/bash # request a "parallel environment" with 2 to 20 cores #$ -pe smp 2-20 # set number of cores for the R cluster to the granted value (between 2 and 20) export MC_CORES=$NSLOTS # we want this: export MC_CLUSTER_TYPE=FORK Rscript /path/to/script.R |

Best Regards

?

Thread (2 messages)

Christian Krause feature request: set default cluster type via environment variable Nov 23 Brian Ripley feature request: set default cluster type via environment variable Nov 24