Skip to content

Parallel Programming

4 messages · TJUN KIAT TEO, Jeff Newmiller, R. Michael Weylandt +1 more

TJUN KIAT TEO
# 
I am trying to do parallel programming and I tried this

library(doSNOW)
library(foreach)

testfunc<-function(x){
x<-x+1
x
}

noc<-2

cl <- makeCluster(do.call(rbind,rep(list("localhost"),noc)), type = "SOCK")
registerDoSNOW(cl)
clusterExport(cl=cl,c("testfunc.r"))


testl<-foreach(pp=1:2) %dopar% {
testfunc(pp)
}


And this works but if I try to enclose my commands inside a text file
to be sourced it doesn't work

noc<-2

testfunc<-function(x){
x<-x+1
x
}

cl <- makeCluster(do.call(rbind,rep(list("localhost"),noc)), type = "SOCK")

registerDoSNOW(cl)

clusterExport(cl=cl,c("a","testfunc.r"))

testl<-foreach(pp=1:2)) %dopar% {
source("test.r")
}
Jeff Newmiller
# 
Then don't do that.

Use your script file to define functions. Source that file before the loop to load them into memory. Call those functions from within your loop.
---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live Go...
                                      Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k
--------------------------------------------------------------------------- 
Sent from my phone. Please excuse my brevity.
Tjun Kiat Teo <teotjunk at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      R. Michael Weylandt
    

    

      
      #
    

  


  

      
On Fri, Sep 21, 2012 at 5:43 AM, Tjun Kiat Teo <teotjunk at gmail.com> wrote:

            

      
      
      
    

        
I'm not sure this is a parallelization issue: when you source() a
file, it doesn't return the last value calculated in quite the way
that running a function directly does.

E.g.,

system("echo 2+2 > test.R")
x <- source("test.R")

identical(x,4) # FALSE

str(x) # Look at what actually returned

source("test.R")[[1]] # Actual result

Perhaps that's tripping you up?

Cheers,
Michael

  
  


  


      

    

    

      
      

        


  

        

      Bert Gunter
    

    

      
      #
    

  


  

      
Inline below.

On Fri, Sep 21, 2012 at 9:05 AM, R. Michael Weylandt

        
<michael.weylandt at gmail.com> wrote:

            

      
      
      
    

        
I believe you are getting confused here.

1) The return of the source() function is a list, not a value, as
Michael's example showed. This appears to be undocumented, however.

2) source() parses and evals the file's contained code in the
specified environment, so, e.g. all assignments are carried out in
that environment, **unlike** functions in which the assignments are
local to the function environment.

e.g. Suppose test.R has the code:
x <- 2

Then:

            

      
      
      
    

        
[1] 2

...  but this last value is **Not** autoprinted, however. As the docs say:

"Note that running code via source differs in a few respects from
entering it at the R command line. Since expressions are not executed
at the top level, auto-printing is not done. "

For these reasons, it seems generally wiser to encapsulate desired
behavior in a function rather than sourcing in a script from a file,
though there are of course issues here, too (e.g. scoping, etc.).

Cheers,
Bert