Using snow on a looping structure

Hi Martin,

Your suggestion really helps! It's exactly what I wanted. I really
appreciate it...

Regarding the array-munging part, the following will do:

b <- aperm(b, c(2,3,4,1))

I have a couple of related issues now:

(1) When running the following I get two warnings on my Mac OS X
10.4.11 (one from each processor, I guess):

cl <- makeCluster(2, type = "SOCK")	

WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME

Why is this warning? How to correct it?

(2) Previously I could follow up the progress of the job by sticking
the following

print(format(Sys.time(), "%D %H:%M:%OS3"))

inside the outermost for loop (with ii index), but now with parallel
computing I couldn't find a similar way to trace the progress. Do you
or anybody know how to do that?

Thanks again,
Gang


On Wed, Dec 3, 2008 at 12:05 PM, Martin Morgan <mtmorgan at fhcrc.org> wrote:

Hi Gang --

"Gang Chen" <gangchen6 at gmail.com> writes:

I'm a newbie running parallel computing, so, sorry for this simple question.

My original code without parallel computing is like this:

runAna <- function(myData, Model, ...) {
      myStat <- wFun(myData, Model, ...)   # myStat: a vector with a
length of nStat
      return(myStat)
}

rStat <- array(0, dim=c(dimx, dimy, dimz, nStat))
for (i in 1:dimx)
for (j in 1:dimy)
for (k in 1:dimz)
     rStat[i, j, k,] <- runAna(rData[i, j, k,], Model, ...)   # each
analysis is on the 4th dimension, and returns nStat numbers which are
stored in the 4th dimension of rStat

I think what you want is along the lines of

a <- array(1:(2*3*4*5), c(2,3,4,5))
b <- apply(a, c(1,2,3), range)

and then as you guessed

library(snow)
cl <- makeCluster(nNodes, type="SOCK")
d <- parApply(cl, a, c(1,2,3), range)
identical(b, d)

[1] TRUE

so for your example, I'd guess

runStat <- parApply(cl, rData, c(1,2,3), runAna, Model=Model)

This is not quite what you want -- the 'result' dimension is the first
rather than last

dim(a)

[1] 2 3 4 5

dim(b)

[1] 2 2 3 4

array-munging is not a speciality of mine, but a simple work-around is
to reorder the dimensions of the original array, so you're applying
to, and writing in, the slice indexed by the first entry

a <- array(1:(5*2*3*4), c(5,2,3,4))
b <- apply(a, c(2,3,4), range)
dim(a)

[1] 5 2 3 4

dim(b)

[1] 2 2 3 4

d <- parApply(cl, a, c(2,3,4), range)

Hope that helps,

Martin

I'm trying to run the above analysis using snow on a machine with two
processors, but could not figure out how to correctly set it up:

nNodes <- 2
library(snow)
cl <- makeCluster(nNodes, type = "SOCK")

I thought I would use parApply, but how should I combine the looping
with parApply? Or no looping at all with something like parApply(cl,
rData, c(1,2,3), ...)?

Thanks in advance,
Gang

--
Martin Morgan
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M2 B169
Phone: (206) 667-2793