I have a Ubuntu Linux 12.04.1 machine running R 2.15.1. I'm trying to run tests building packages, so install.packages() is trying to install about about 1000 packages from source. I'm running into some strange issues that seem related to using values for Ncpus other than 1. When I use Ncpus=32, about 260 of the packages fail to install. When I tried again with Ncpus=2, more of them get installed, and there were only 234 failed packages. I tried it again with Ncpus=1, after which there were only 142 failed packages. (I think these remaining failed packages are due to missing external dependencies, like RODBC, Java, and various development libraries, so these aren't a concern at the moment.) In the code for install.packages, I see that if Ncpus>1, it passes the Ncpus to make, as in 'make -k -j 32'. Is it possible that these packages are failing because of this option to make? Has anyone else run into this issue before? Thanks! -Winston
Problems with install.packages when Ncpus > 1
4 messages · Winston Chang, Brian Ripley, Uwe Ligges
On 06/10/2012 05:42, Winston Chang wrote:
I have a Ubuntu Linux 12.04.1 machine running R 2.15.1. I'm trying to run tests building packages, so install.packages() is trying to install about about 1000 packages from source. I'm running into some strange issues that seem related to using values for Ncpus other than 1. When I use Ncpus=32, about 260 of the packages fail to install. When I tried again with Ncpus=2, more of them get installed, and there were only 234 failed packages. I tried it again with Ncpus=1, after which there were only 142 failed packages. (I think these remaining failed packages are due to missing external dependencies, like RODBC, Java, and various development libraries, so these aren't a concern at the moment.) In the code for install.packages, I see that if Ncpus>1, it passes the Ncpus to make, as in 'make -k -j 32'. Is it possible that these packages are failing because of this option to make?
It is perfectly correct option: please do your homework before posting as the posting guide asked of you.
Has anyone else run into this issue before?
Yes. It is due to missing dependencies between packages. I find this works well on CRAN *provided* that the BioC repositories are also selected so that dependencies can be traced via BioC dependencies. However, BioC does have a fair few missing dependencies in its packages (run R CMD check over BioC to see them). Take a look at the logs of the failed packages. One I see frequently is that ddgraph fails because it depends indirectly on RBGL which takes a long time to install and is still being done.
Thanks! -Winston
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
In the code for install.packages, I see that if Ncpus>1, it passes the Ncpus to make, as in 'make -k -j 32'. Is it possible that these packages are failing because of this option to make?
It is perfectly correct option: please do your homework before posting as the posting guide asked of you.
The question isn't about whether these are valid make options. I realize that '-j <n>' is a valid option for make, just like Ncpus is a valid option for install.packages. The question is whether that option is unexpectedly resulting in failed compiles for some packages.
Has anyone else run into this issue before?
Yes. It is due to missing dependencies between packages. I find this works well on CRAN *provided* that the BioC repositories are also selected so that dependencies can be traced via BioC dependencies. However, BioC does have a fair few missing dependencies in its packages (run R CMD check over BioC to see them). Take a look at the logs of the failed packages. One I see frequently is that ddgraph fails because it depends indirectly on RBGL which takes a long time to install and is still being done.
Thanks, I'll take a closer look at these those cross dependencies. -Winston
On 06.10.2012 18:02, Winston Chang wrote:
In the code for install.packages, I see that if Ncpus>1, it passes the Ncpus to make, as in 'make -k -j 32'. Is it possible that these packages are failing because of this option to make?
It is perfectly correct option: please do your homework before posting as the posting guide asked of you.
The question isn't about whether these are valid make options. I realize that '-j <n>' is a valid option for make, just like Ncpus is a valid option for install.packages. The question is whether that option is unexpectedly resulting in failed compiles for some packages.
I think Brian answered this already: Not unexpectedly. It fails in case when a package installation requires another package and that is installed at the same time or not yet installed. Such things happen when install time dependencies are not correctly declared. R CMD check finds (at least most of) such declaration errors nowadays. Uwe Ligges
Has anyone else run into this issue before?
Yes. It is due to missing dependencies between packages. I find this works well on CRAN *provided* that the BioC repositories are also selected so that dependencies can be traced via BioC dependencies. However, BioC does have a fair few missing dependencies in its packages (run R CMD check over BioC to see them). Take a look at the logs of the failed packages. One I see frequently is that ddgraph fails because it depends indirectly on RBGL which takes a long time to install and is still being done.
Thanks, I'll take a closer look at these those cross dependencies. -Winston
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel