Hi all, I can't get the functionality of the package parallel to work. Specifically, makeCluster() hangs when I run it. I first noticed the problem when trying to run Rstan with multiple cores and the traced it back to the core package parallel. The following results in R hanging after the call to makeCluster. library(parallel) # Calculate the number of cores no_cores <- detectCores() - 1 # Initiate cluster cl <- makeCluster(no_cores) I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017 laptop with 4 cores. platform?????? x86_64-apple-darwin15.6.0 arch?????????? x86_64 os???????????? darwin15.6.0 system???????? x86_64, darwin15.6.0 status major????????? 3 minor????????? 4.3 year?????????? 2017 month????????? 11 day??????????? 30 svn rev??????? 73796 language?????? R version.string R version 3.4.3 (2017-11-30) nickname?????? Kite-Eating Tree The problem replicates in R --vanilla I've spent hours googling for solutions but can't find any reports of this problem. Any help would be appreciated. Florian
makeCluster hangs
4 messages · T. Florian Jaeger, Henrik Bengtsson, Tomas Kalibera
A few quick comments: * You mention R --vanilla, but make sure to try with parallel::makeCluster(), so that you don't happen to pick up snow::makeCluster() if 'snow' is attached and ahead of parallel on the search() path. * Try creating a single background worker, i.e. parallel::makeCluster(1L). * Try with cl <- future::makeClusterPSOCK(1L, verbose = TRUE), which gives the same thing, but it also show you some details on what it does internally; that may give some clues where it stalls. /Henrik On Sat, Feb 10, 2018 at 12:11 PM, T. Florian Jaeger
<fjaeger at ur.rochester.edu> wrote:
Hi all, I can't get the functionality of the package parallel to work. Specifically, makeCluster() hangs when I run it. I first noticed the problem when trying to run Rstan with multiple cores and the traced it back to the core package parallel. The following results in R hanging after the call to makeCluster. library(parallel) # Calculate the number of cores no_cores <- detectCores() - 1 # Initiate cluster cl <- makeCluster(no_cores) I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017 laptop with 4 cores. platform x86_64-apple-darwin15.6.0 arch x86_64 os darwin15.6.0 system x86_64, darwin15.6.0 status major 3 minor 4.3 year 2017 month 11 day 30 svn rev 73796 language R version.string R version 3.4.3 (2017-11-30) nickname Kite-Eating Tree The problem replicates in R --vanilla I've spent hours googling for solutions but can't find any reports of this problem. Any help would be appreciated. Florian
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Dear Henrik, thank you, for the quick reply. Bizarrely enough, the problem vanished when I woke the computer from sleep (I had previously replicated the problem after several restarts of both R and the MacOS). I will follow-up if I can again replicate the problem. Florian
On 2/10/18 4:39 PM, Henrik Bengtsson wrote:
A few quick comments: * You mention R --vanilla, but make sure to try with parallel::makeCluster(), so that you don't happen to pick up snow::makeCluster() if 'snow' is attached and ahead of parallel on the search() path. * Try creating a single background worker, i.e. parallel::makeCluster(1L). * Try with cl <- future::makeClusterPSOCK(1L, verbose = TRUE), which gives the same thing, but it also show you some details on what it does internally; that may give some clues where it stalls. /Henrik On Sat, Feb 10, 2018 at 12:11 PM, T. Florian Jaeger <fjaeger at ur.rochester.edu> wrote:
Hi all, I can't get the functionality of the package parallel to work. Specifically, makeCluster() hangs when I run it. I first noticed the problem when trying to run Rstan with multiple cores and the traced it back to the core package parallel. The following results in R hanging after the call to makeCluster. library(parallel) # Calculate the number of cores no_cores <- detectCores() - 1 # Initiate cluster cl <- makeCluster(no_cores) I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017 laptop with 4 cores. platform x86_64-apple-darwin15.6.0 arch x86_64 os darwin15.6.0 system x86_64, darwin15.6.0 status major 3 minor 4.3 year 2017 month 11 day 30 svn rev 73796 language R version.string R version 3.4.3 (2017-11-30) nickname Kite-Eating Tree The problem replicates in R --vanilla I've spent hours googling for solutions but can't find any reports of this problem. Any help would be appreciated. Florian
______________________________________________ R-devel at r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=O6dqVFPEDpdoXY3wkv8u6o0LHKx4WbQ_itn0O87jj5s&m=R7DIWWqYTP2xarhrvKcymtN3XlAQ9vHLFDhPL6FxQ60&s=7F8Lez-XE8iC2JBU4JYsEtF3U0HObhMnCASud5xTgNM&e=
.
T. Florian Jaeger Professor of the Brain and Cognitive Sciences, Computer Science Alfred P. Sloan Research Fellow University of Rochester, Meliora Hall, Box 270268, Rochester, NY 14627-0268 USA P: +1 (585) 276 3611 F: +1 (585) 442 9216 E: fjaeger at ur.rochester.edu B: http://www.hlplab.wordpress.com/ T: https://twitter.com/_hlplab_ To schedule an appointment, please visit http://www.bcs.rochester.edu/people/fjaeger/appointments.html [[alternative HTML version deleted]]
Also using R-devel might help - the forking support in parallel has been made more robust against race conditions, but the changes are probably too substantial to port to 3.4.x. If you find how to cause a race condition using parallel/forking in R-devel, a report would be greatly appreciated. Tomas
On 02/11/2018 09:51 PM, T. Florian Jaeger wrote:
Dear Henrik, thank you, for the quick reply. Bizarrely enough, the problem vanished when I woke the computer from sleep (I had previously replicated the problem after several restarts of both R and the MacOS). I will follow-up if I can again replicate the problem. Florian On 2/10/18 4:39 PM, Henrik Bengtsson wrote:
A few quick comments: * You mention R --vanilla, but make sure to try with parallel::makeCluster(), so that you don't happen to pick up snow::makeCluster() if 'snow' is attached and ahead of parallel on the search() path. * Try creating a single background worker, i.e. parallel::makeCluster(1L). * Try with cl <- future::makeClusterPSOCK(1L, verbose = TRUE), which gives the same thing, but it also show you some details on what it does internally; that may give some clues where it stalls. /Henrik On Sat, Feb 10, 2018 at 12:11 PM, T. Florian Jaeger <fjaeger at ur.rochester.edu> wrote:
Hi all, I can't get the functionality of the package parallel to work. Specifically, makeCluster() hangs when I run it. I first noticed the problem when trying to run Rstan with multiple cores and the traced it back to the core package parallel. The following results in R hanging after the call to makeCluster. library(parallel) # Calculate the number of cores no_cores <- detectCores() - 1 # Initiate cluster cl <- makeCluster(no_cores) I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017 laptop with 4 cores. platform x86_64-apple-darwin15.6.0 arch x86_64 os darwin15.6.0 system x86_64, darwin15.6.0 status major 3 minor 4.3 year 2017 month 11 day 30 svn rev 73796 language R version.string R version 3.4.3 (2017-11-30) nickname Kite-Eating Tree The problem replicates in R --vanilla I've spent hours googling for solutions but can't find any reports of this problem. Any help would be appreciated. Florian
______________________________________________ R-devel at r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=O6dqVFPEDpdoXY3wkv8u6o0LHKx4WbQ_itn0O87jj5s&m=R7DIWWqYTP2xarhrvKcymtN3XlAQ9vHLFDhPL6FxQ60&s=7F8Lez-XE8iC2JBU4JYsEtF3U0HObhMnCASud5xTgNM&e=
.