Skip to content

cluster error with clusterR and calc

4 messages · Guillaume Drolet, Robert J. Hijmans

Original
Guillaume Drolet
# 
Hi,I have the piece of code below which returns this error: Error in
clusterR(s, fun1, filename = f) : cluster error# Piece of code# x, y, z are
rasters# f is a filenamefun1 <- function(x) {      calc(x, fun = function(x)
{        10 * (x[2] - x[3]) / ENV.LAPSERATE + x[1]      })}s <- stack(x, y,
z)beginCluster(nodes)  h <- clusterR(s, fun1, filename = f)endCluster()# End
piece of codeYesterday I got the code to run without error by first
detaching package "parallel" detach("package:parallel", unload = TRUE) [It
seems that package "parallel" is masking functions from package "snow" which
is used by clusterR]. Today I just can't detach "parallel" (Error in
detach("package:parallel") : argument 'name' incorrect). This may be due to
the fact that I updated a buch of packages yesterday.If you have any idea
about what is causing this error, please let me know. I'll be very
happy!Thanks,Guillaume



--
View this message in context: http://r-sig-geo.2731867.n2.nabble.com/cluster-error-with-clusterR-and-calc-tp7585148.html
Sent from the R-sig-geo mailing list archive at Nabble.com.
Robert J. Hijmans
# 
Guillaume,

You cannot use an unevaluated variable in the clusterR "fun" function
as such variables are not exported to the nodes. It works by replacing
ENV.LAPSERATE with a number.
Also, note my different (matrix) notation in the function for calc,
but that for cases like this, where you want to refer to specific
layers, the overlay function is more convenient.

Robert

library(raster)
x <- raster()
y = setValues(x, runif(ncell(x)))
z = setValues(x, runif(ncell(x)))
x = setValues(x, runif(ncell(x)))
f= 'test.tif'
s <- stack(x, y, z)

# note, I replaced ENV.LAPSERATE with a number
fun1 <- function(x) calc(x, fun = function(d){ 10 * (d[,2] - d[,3]) /
6 + d[,1] })
fun2 <- function(x) overlay(x, fun = function(d,e,f){ 10 * (e - f) / 6 + d })

beginCluster(4)
h1 <- clusterR(s, fun1)
h2 <- clusterR(s, fun2)
# or like this
h3 <- clusterR(s, overlay, args=list(fun = function(d,e,f){ 10 * (e -
f) / 6 + d }))
endCluster()

On Wed, Nov 20, 2013 at 8:40 AM, Guillaume Drolet
<droletguillaume at gmail.com> wrote:
Guillaume Drolet
# 
Hi Robert,

Thanks for your help. I thought that overlay didn't work with clusterR (see
?clusterR: ... For example, it works with calc but not with overlay.)

Is there any way to export variables to the clusters when using clusterR,
similar to when using clusterExport() with snow, or if it HAS to be hard
coded?

Best wishes,

Guillaume



--
View this message in context: http://r-sig-geo.2731867.n2.nabble.com/cluster-error-with-clusterR-and-calc-tp7585148p7585153.html
Sent from the R-sig-geo mailing list archive at Nabble.com.
Robert J. Hijmans
# 
The manual is wrong. It should say that clusterR also works with
overlay if you use a single RasterStack or RasterBrick object as the
first argument (it would not work if you provided three RasterLayer
objects)

Yes, you can use clusterExport, by first calling getCluster() to get
the cluster object.

library(raster)
x <- raster()
y = setValues(x, runif(ncell(x)))
z = setValues(x, runif(ncell(x)))
x = setValues(x, runif(ncell(x)))
f= 'test.tif'
s <- stack(x, y, z)

ENV.LAPSERATE = 6
fun1 <- function(x) calc(x, fun = function(d){ 10 * (d[,2] - d[,3]) /
ENV.LAPSERATE + d[,1] })
fun2 <- function(x) overlay(x, fun = function(d,e,f){ 10 * (e - f) /
ENV.LAPSERATE + d })

beginCluster(4)
cl <- getCluster()
clusterExport(cl, "ENV.LAPSERATE")
h1 <- clusterR(s, fun1)
h2 <- clusterR(s, fun2)
h3 <- clusterR(s, overlay, args=list(fun = function(d,e,f){ 10 * (e -
f) / ENV.LAPSERATE + d }))
endCluster()

On Wed, Nov 20, 2013 at 9:49 AM, Guillaume Drolet
<droletguillaume at gmail.com> wrote: