Skip to content

parallelize distance matrix

3 messages · Ariel Fuentesdi, Roman Luštrik, Vijay Lulla

Ariel Fuentesdi
# 
Hi, I want to parallelize the calculation of a distance matrix, I've tried
but I wasn't succesful, this is what I did with a training data.

My data set is:

ll <- matrix(c(5, 6, 60, 60), ncol=2)

And I use the function *spDistsN1* from the library *"sp"* to obtain a
distance matrix with *apply*:

apply(ll, 1, function(x) spDistsN1(as.matrix(ll), x, longlat = T))

But I want to do it with parallelization, so for that:

library(parallel)
ncore <- detectCores()
cl <- makeCluster(ncore)
clusterEvalQ(cl = cl, expr = c(library(sp)))
parRapply(cl = cl, x = ll, FUN =  function(x) spDistsN1(as.matrix(ll), x,
longlat = T))

It shows the following error:

*Error in checkForRemoteErrors(val) : 4 nodes produced errors; first error:
object 'll' not found*

How do I fix it?
Regards,
Ariel Fuentes
Roman Luštrik
# 
Cross posted on SO:
https://stackoverflow.com/questions/51952776/parallelization-apply-to-parrapply

On Tue, Aug 21, 2018 at 8:15 PM Ariel Fuentesdi <ariel.fuentesdi at usach.cl>
wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Vijay Lulla
    

    

      
      #
    

  


  

      
You'll have to do clusterExport(cl, c("ll")) before you call parRapply.
Please see ?`parRapply` and pages 10 and 11 of the parallel::parallel
vignette.
HTH,
Vijay.

On Tue, Aug 21, 2018 at 2:54 PM Roman Lu?trik <roman.lustrik at gmail.com>
wrote: