Skip to content
Prev 787 / 2152 Next

Serializing global state from within a namespace to distribute to other workers

On Tue, Nov 9, 2010 at 12:50 PM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
The print() statement for functions will tell you if an environment is
associated with the function that will need to be serialized.

I ended up in gdb and sprinkling Rprintf's around serialize.c and
loadsave.c to try to understand this better.  In the end I changed the
assign statement to use substitute to get around the fact that a
NAMESPACE is associated with FUN no matter how hard I try to strip it
off :

parallelapply <- function(x, FUN, ...) {
 environment(FUN) <- .GlobalEnv   # does not have intended effect
 assign(".GLOBAL.FUN",
              eval.parent(substitute(function(y) { FUN(y, ...) })),
              env=.GlobalEnv)
 environment(.GLOBAL.FUN) <- .GlobalEnv   # does not have intended effect
 save(list = ls(envir = .GlobalEnv, all.names = TRUE),
      file = "/tmp/.Rdata",
 # Here we distribute the .Rdata file to other workers that load it
and then run .GLOBAL.FUN(n)
 # this works great if parallelapply is in a package without a
NAMESPACE, but fails on loadNamespace otherwise.
}

              - Murray