Skip to content

Snow on Windows Cluster

2 messages · Markus Schmidberger, Luke Tierney

Original
Markus Schmidberger
# 
Hello,

the package snow is not working on a windows cluster with MPICH2 and 
Rmpi. There is an error in makeCluster:

launch failed: CreateProcess(/usr/bin/env 
"RPROG="C:\Programme\R\R-2.5.1\bin\R" "OUT=/dev/null" "R_LIBS=" 
C:/Programme/R/R-2.5.1/library/snow/RMPInode.sh) on 'cl1' failed, error 
3 - Das System kann den angegbenen Pfad nicht finden.

I looked into makeMPIcluster. It could not work, mpi.comm.spawn gets the 
shell command for unix to start the slaves. A simple solution is to use 
the windows-code from mpi.spawn.rslaves, then it looks like this. And is 
working on our windows-cluster. (small failures on stopping cluster)

# Fro windows
if (.Platform$OS=="windows"){
    workdrive <- unlist(strsplit(getwd(),":"))[1]
    tmpdrive <- unlist(strsplit(tempdir(),":"))[1]
    worktmp <- as.logical(toupper(workdrive)==toupper(tmpdrive))
    tmpdir <- unlist(strsplit(tempdir(),"/Rtmp"))[1]
    localhost <- Sys.getenv("COMPUTERNAME")
    networkdrive <-.Call("RegQuery", 
as.integer(2),paste("NETWORK\\",workdrive,sep=""),
                       PACKAGE="Rmpi")
    remotepath <-networkdrive[which(networkdrive=="RemotePath")+1]
    mapdrive=TRUE
    mapdrive <- as.logical(mapdrive && !is.null(remotepath))
    Rscript=system.file("slaveload.R", package="snow")
    arg <- c(Rscript, R.home(), workdrive, getwd(),worktmp, tmpdir,
                   localhost, mapdrive, remotepath)
            
#Rscript,R_HOME,WDrive,WDir,WorkTmp,TmpDir,Master,MapDrive,RemotePath
    print("super")
    count<-mpi.comm.spawn(
        slave=system.file("Rslaves.bat", package="Rmpi"),
        slavearg=arg,
        nslaves=count)
} else{   # for unix
    count <- mpi.comm.spawn(slave = "/usr/bin/env",
                                slavearg = args,
                                nslaves = count,
                                intercomm = intercomm)
}

Doing some work, there should be a better and nicer solution with 
directly using mpi.spawn.Rslaves.

Best
Markus
7 days later
Luke Tierney
# 
Thanks for the note.

Could you give me more info on the "small failures on stopping cluster"?

I don't currently have a windows system I can test this on so I can't
fold it into the snow release yet. But hopefully I can in the next
month or so. I may get back in touch at that point.

Meanwhile I have been working with a modified shell script that can be
used with mpirun under lam on unix.  I've attached the current
version.  It may turn out that this approach is a little more
maitainable on the Windows side as well.  The way this gets used is

     mpirun -np 5 RMPISNOW

to run a master and 4 workers, and then in the master

     cl <- getMPIcluster()

instead of using makeCluster or makeMPIcluster.  I have not yet found
the MICH2 analog of the LAMRANK environment variable but suspect
something similar does exist.

Best,

luke
On Mon, 3 Sep 2007, Markus Schmidberger wrote: