Dear Paul, I observe the some behavior with an almost identical setting (OpenMPI 1.4.3, R 2.14.1, parallel, snow 0.3.8, Rmpi 0.5-9). This seems to be a NAMESPACE issue in snow. It does not define any S3 methods, it rather exports all functions "as is". This works just fine if snow is attached but fails for S3 methods if not. I cc'ed the maintainer of snow, Luke Tierney, to let him know. Best, Stefan
On 01/11/2012 12:00 PM, r-sig-hpc-request at r-project.org wrote:
Send R-sig-hpc mailing list submissions to r-sig-hpc at r-project.org To subscribe or unsubscribe via the World Wide Web, visit
https://stat.ethz.ch/mailman/listinfo/r-sig-hpc or, via email, send a message with subject or body 'help' to r-sig-hpc-request at r-project.org You can reach the person managing the list at r-sig-hpc-owner at r-project.org When replying, please edit your Subject line so it is more specific than "Re: Contents of R-sig-hpc digest..." Today's Topics: 1. R parallel package<-> stopCluster method needed for mpi cluster (Paul Johnson) ---------------------------------------------------------------------- Message: 1 Date: Tue, 10 Jan 2012 22:21:30 -0600 From: Paul Johnson<pauljohn32 at gmail.com> To: R SIG High Performance Computing<r-sig-hpc at r-project.org> Subject: [R-sig-hpc] R parallel package<-> stopCluster method needed for mpi cluster Message-ID: <CAErODj9Bzt2ytn_maY0j4LwbzRU0ibtFQAAbu4q0HOodZSwgaQ at mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 hi, everybody in r-sig-hpc land. hopefully this year is bringing you good news. I'm trying to digest the fact that R now has a parallel package. I'm pretty sure I've run into something that is on the TODO list. Briefly, the stopCluster method from R parallel doesn't lead back to the stopCluster methods from snow to shut down an MPI cluster. Now, the long version: We have an OpenMPI based cluster, I've been working up examples that use Rmpi, snow, snowft, and now R parallel. Here's my parallel effort. http://winstat.quant.ku.edu/svn/hpcexample/trunk/Ex65-R-parallel Its just a submission script and an example cluster usage. I'm stuck on (at least) one detail. If you examine http://winstat.quant.ku.edu/svn/hpcexample/trunk/Ex65-R-parallel/parallel-hello.R you see the cluster is created like so library(parallel) ## Causes the snow package's MPI support to be invoked, cl<- makeCluster(10, type="MPI") That actually seems to be fine. the parallel makeCluster method refers the work to snow::makeCluster. Works fine. After that, how to close the thing down? When I run stopCluster(cl) processing crashes, and the error output has this: Loading required package: Rmpi Error in UseMethod("stopCluster") : no applicable method for 'stopCluster' applied to an object of class "c('spawnedMPIcluster', 'MPIcluster', 'cluster')" Calls:<Anonymous> Execution halted -------------------------------------------------------------------------- orterun has exited due to process rank 0 with PID 28684 on node compute-2-11.local exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by orterun (as reported here). -------------------------------------------------------------------------- I *think* I need to call the stopCluster method from snow. It seems like this should have worked: snow::stopCluster(cl) or this: snow::stopCluster.MPIcluster(cl) but it gives the same error. But if I load the snow package, all is good. library(snow) stopCluster(cl) mpi.quit() The error log has a warning, but otherwise it appears OK. Attaching package: 'snow' The following object(s) are masked from 'package:parallel': clusterApply, clusterApplyLB, clusterCall, clusterEvalQ, clusterExport, clusterMap, clusterSplit, makeCluster, parApply, parCapply, parLapply, parRapply, parSapply, splitIndices, stopCluster What do you think. This is a known issue? Should I bother somebody about it? I mean somebody besides you, of course.