foreach error checking (when a cluster stops)
Hi Jonathan, Sorry, there's is nothing like that in foreach. The functions that are used to provide information about the parallel backend include: getDoParWorkers() getDoParName() getDoParVersion() getDoParRegistered() but no "health check" functions. It's a good idea, though. I believe they include that sort of thing in database frameworks. By the way, I had no idea that you could use the doSNOW package with snowfall, although I guess it makes sense since they give you access to the cluster object. But I've only ever tested doSNOW with snow. - Steve
On Sat, Feb 2, 2013 at 12:19 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
R-sig-hpc'ers:
I've been modding some code lately to take advantage of the foreach
generalization (so I don't have to write multiple statements depending on
the parallel backend that I have running), but I'm trying to figure out how
to check for a registered, but no longer running, parallel backend, e.g.:
require("foreach")
require("snowfall")
require("doSNOW")
# Start a snowfall cluster and register it with foreach
sfInit(cpu=2,parallel=TRUE)
cl <- sfGetCluster()
registerDoSNOW(cl)
# Some foreach statement:
d <- data.frame(x=1:10, y=rnorm(10))
s <- foreach(d=iter(d, by='row'), .combine=rbind) %dopar% d
# Works fine. Now stop the cluster:
sfStop()
s <- foreach(d=iter(d, by='row'), .combine=rbind) %dopar% d
# Doesn't work, because the snow cluster is gone.
# I can manually set this, but is there some test I can do
# to test for whether the cluster is working and, if not:
registerDoSEQ()
s <- foreach(d=iter(d, by='row'), .combine=rbind) %dopar% d
# e.g.
if(checkForWorkingBackend == FALSE)
{
registerDoSEQ()
}
--
Jonathan A. Greenberg, PhD
Assistant Professor
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
607 South Mathews Avenue, MC 150
Urbana, IL 61801
Phone: 217-300-1924
http://www.geog.illinois.edu/~jgrn/
AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
[[alternative HTML version deleted]]
_______________________________________________ R-sig-hpc mailing list R-sig-hpc at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-hpc