Dear all, I am looking for some way on how to parallelize some computation on a vectorized function. I have a custom made vectorized function and also have a very length vector (of length some 2 million entries.) There are equivalents for all apply family of functions however could not find any way how to apply some parallel computation on a single vectorized function. I have tried with making my vector to list (of same length) and then applying sfLapply() function. However this process is actually taking more time. I am using windows vista machine and snowfall package. Any help will be highly appreciated. Thanks,
Parallel computation on vectorized function
4 messages · Brian G. Peterson, Megh Dal, Zachary Mayer
On Sat, 2011-08-27 at 07:41 -0700, Megh Dal wrote:
Dear all, I am looking for some way on how to parallelize some computation on a vectorized function. I have a custom made vectorized function and also have a very length vector (of length some 2 million entries.) There are equivalents for all apply family of functions however could not find any way how to apply some parallel computation on a single vectorized function. I have tried with making my vector to list (of same length) and then applying sfLapply() function. However this process is actually taking more time. I am using windows vista machine and snowfall package.
take a look at the iterators package, it can chunk up your vector. I use it with foreach, but there's no reason you couldn't use it to chunk something up in snowfall.
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock
Thanks Brian for your reply. Would you mind to give me an example how you have achieved that with foreach package? I could then try with the snowfall package in windows. Thanks, ----- Original Message ----- From: Brian G. Peterson <brian at braverock.com> To: Megh Dal <megh700004 at yahoo.com> Cc: "r-sig-hpc at r-project.org" <r-sig-hpc at r-project.org> Sent: Saturday, August 27, 2011 8:14 PM Subject: Re: [R-sig-hpc] Parallel computation on vectorized function
On Sat, 2011-08-27 at 07:41 -0700, Megh Dal wrote:
Dear all, I am looking for some way on how to parallelize some computation on a vectorized function. I have a custom made vectorized function and also have a very length vector (of length some 2 million entries.) There are equivalents for all apply family of functions however could not find any way how to apply some parallel computation on a single vectorized function. I have tried with making my vector to list (of same length) and then applying sfLapply() function. However this process is actually taking more time. I am using windows vista machine and snowfall package.
take a look at the iterators package, it can chunk up your vector. I use it with foreach, but there's no reason you couldn't use it to chunk something up in snowfall.
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock
See this question on stack overflow: http://stackoverflow.com/questions/5571774/what-is-the-easiest-way-to-parallelize-a-vectorized-function-in-r "Have a look at the pvec function in multicore" Sent from my iPhone
On Aug 27, 2011, at 10:41 AM, Megh Dal <megh700004 at yahoo.com> wrote:
Dear all, I am looking for some way on how to parallelize some computation on a vectorized function. I have a custom made vectorized function and also have a very length vector (of length some 2 million entries.) There are equivalents for all apply family of functions however could not find any way how to apply some parallel computation on a single vectorized function. I have tried with making my vector to list (of same length) and then applying sfLapply() function. However this process is actually taking more time. I am using windows vista machine and snowfall package. Any help will be highly appreciated. Thanks,
_______________________________________________ R-sig-hpc mailing list R-sig-hpc at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-hpc