Skip to content

Error in clusterR

3 messages · Srinivas V, Jonathan Greenberg

#
Hi,

I wrote a code which uses clusterR to run a function on multiple CPUs to 
process MODIS 16 day dataset. The function converts a brick to 
dataframe. The existing commands to do this in package raster is very 
slow on large datasets, therefore I'm attempting to use the multicore 
option to speed up the process.

I get the following error
Error in clusterR(n1, qckextract) : cluster error

I would appreciate your advice on to fix the function. Thanks!


library(raster)

n1<-brick("ndvih25v08masked.grd")

beginCluster(nice=20)
nodes <- length(getCluster())
bs <- blockSize(n1, minblocks=nodes*4)

qckextract <- function(i) {
   for (i in 1:length(bs$n)) {
     e<- extent(xmin(x), xmax(x), yFromRow(x, bs$row[i]+bs$nrows[i]-1) - 
0.5 * yres(x), yFromRow(x, bs$row[i])+0.5 * yres(x))
     n <- cellsFromExtent(x,e)
     v <- (getValues(x, bs$row[i], bs$nrows[i]))/10000
     v<-as.data.frame(v)
     v<- as.data.frame(cbind(n,v))
   }
   return(v)
}

nx<-clusterR(n1,qckextract)
endCluster()
#
Srinivas:

Can you explain, more generally, what you are trying to accomplish?
It seems like you are just trying to get a list-of-dataframes from the
MODIS pixels.  It might be easier to do something like this with the
package "foreach".  Keep in mind reading/write is often I/O limited,
so you may not see any performance gain reading in parallel.

I'd also recommend looking at "rasterEngine" in my spatial.tools
package.  Here's a tutorial on how to use it:

http://publish.illinois.edu/jgrn/software-and-datasets/rasterengine-tutorial/

--j
On Tue, May 13, 2014 at 5:11 AM, Srinivas V <srinivasv at feralindia.org> wrote:

  
    
6 days later