Problems parallelizing glmnet
On 09/06/2012 12:20 PM, Zachary Mayer wrote:
I was not aware of this unfortunate limitation. Was it specific to a particular back-end, or just foreach in general? On Thu, Sep 6, 2012 at 1:15 PM, Patrik Waldmann <patrik.waldmann at boku.ac.at>wrote:
I would like to avoid foreach since we showed earlier that it is VERY slow.
Foreach is 'very slow' (i.e. has a lot of overhead) if your function evaluation to be parallelized in computationally trivial, in which case the benefits of parallelization are rather questionable. In the case where your function evaluation is not trivial, the overhead of foreach is marginal in comparison to the various *apply* methods that it wraps, and almost certainly worth the flexibility to not change code as your parallel backend changes.
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock