Hi all,
I'm wondering if/where I can go to tell R to use a specific version of
my gcc compiler when installing packages from source, via
`install.packages("...", type='source')` or when running `R CMD
INSTALL some_source_packge.tar.gz` from the command line.
The reason I ask is because I've downloaded gcc-4.4 from HPC/OSX[1],
and it unfortunately puts the compilers in /usr/local/bin/gcc, w/o
version info (it would be nice if it were /usr/local/bin/gcc-4.4). R
tries to use this when compiling, but these compilers do not support
the "-arch=..." flags, so it bombs.
I've been hunting around in the tools:::.install_packges code and also
the "R Installation and Adminstration"[2] document, but can't find
what I'm looking for. I have a feeling that perhaps I can put
something into my ~/.R/Makevars, or ~/.R/Makevars-i386-apple-
darwin8.11.1 file, but I'm just taking stabs at the dark, so not
having much luck.
I can work around this by temporarily removing /usr/local/bin from my
path when trying to `R CMD INSTALL` packages by source, but I was
hoping there might be a more elegant solution. The fact that
installing by source gets the `make` command from the system
environment makes me a bit hopeful, but I'm not really sure.
Thanks for any help,
-steve
[1] HPC/OSX: http://hpc.sourceforge.net/
[2] R Install/admin: http://cran.r-project.org/doc/manuals/R-admin.html#Linear-algebra
--
Steve Lianoglou
Graduate Student: Physiology, Biophysics and Systems Biology
Weill Medical College of Cornell University
http://cbio.mskcc.org/~lianos
Compiling packages with specific gcc
7 messages · Duncan Murdoch, Kasper Daniel Hansen, Simon Urbanek +1 more
On 5/5/2009 1:05 PM, Steve Lianoglou wrote:
Hi all,
I'm wondering if/where I can go to tell R to use a specific version of
my gcc compiler when installing packages from source, via
`install.packages("...", type='source')` or when running `R CMD
INSTALL some_source_packge.tar.gz` from the command line.
The reason I ask is because I've downloaded gcc-4.4 from HPC/OSX[1],
and it unfortunately puts the compilers in /usr/local/bin/gcc, w/o
version info (it would be nice if it were /usr/local/bin/gcc-4.4). R
tries to use this when compiling, but these compilers do not support
the "-arch=..." flags, so it bombs.
I've been hunting around in the tools:::.install_packges code and also
the "R Installation and Adminstration"[2] document, but can't find
what I'm looking for. I have a feeling that perhaps I can put
something into my ~/.R/Makevars, or ~/.R/Makevars-i386-apple-
darwin8.11.1 file, but I'm just taking stabs at the dark, so not
having much luck.
I can work around this by temporarily removing /usr/local/bin from my
path when trying to `R CMD INSTALL` packages by source, but I was
hoping there might be a more elegant solution. The fact that
installing by source gets the `make` command from the system
environment makes me a bit hopeful, but I'm not really sure.
Thanks for any help,
You can pass configure args with the configure.args parameter to install.packages. I'm not sure how many you need to set, but you'll likely need to set CC, CPP, CXX, F77, FC, OBJC, and maybe some of the associated flags. You can see the list using "R CMD config" in the console. Duncan Murdoch
You will want to recompile R with the new compiler. Then, whenever you compile a package, it will use the same compiler as R was compiled with. Mixing compilers might be possible using the hints from Duncan, but I am pretty sure it is discouraged. In the past Simon has discouraged use of the HPC compilers. I don't remember the reasons, but I respect the source of the information :) He knows way more about mac compilers than I do. Of course, this was a while back and things might have changed. Kasper
On May 5, 2009, at 10:05 , Steve Lianoglou wrote:
Hi all,
I'm wondering if/where I can go to tell R to use a specific version
of my gcc compiler when installing packages from source, via
`install.packages("...", type='source')` or when running `R CMD
INSTALL some_source_packge.tar.gz` from the command line.
The reason I ask is because I've downloaded gcc-4.4 from HPC/OSX[1],
and it unfortunately puts the compilers in /usr/local/bin/gcc, w/o
version info (it would be nice if it were /usr/local/bin/gcc-4.4). R
tries to use this when compiling, but these compilers do not support
the "-arch=..." flags, so it bombs.
I've been hunting around in the tools:::.install_packges code and
also the "R Installation and Adminstration"[2] document, but can't
find what I'm looking for. I have a feeling that perhaps I can put
something into my ~/.R/Makevars, or ~/.R/Makevars-i386-apple-
darwin8.11.1 file, but I'm just taking stabs at the dark, so not
having much luck.
I can work around this by temporarily removing /usr/local/bin from
my path when trying to `R CMD INSTALL` packages by source, but I was
hoping there might be a more elegant solution. The fact that
installing by source gets the `make` command from the system
environment makes me a bit hopeful, but I'm not really sure.
Thanks for any help,
-steve
[1] HPC/OSX: http://hpc.sourceforge.net/
[2] R Install/admin: http://cran.r-project.org/doc/manuals/R-admin.html#Linear-algebra
--
Steve Lianoglou
Graduate Student: Physiology, Biophysics and Systems Biology
Weill Medical College of Cornell University
http://cbio.mskcc.org/~lianos
_______________________________________________ R-SIG-Mac mailing list R-SIG-Mac at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Thanks Duncan & Kasper, I've been able to squeeze out of my problem ... Duncan:
You can pass configure args with the configure.args parameter to install.packages. I'm not sure how many you need to set, but you'll likely need to set CC, CPP, CXX, F77, FC, OBJC, and maybe some of the associated flags. You can see the list using "R CMD config" in the console.
The R CMD config was the trick. I actually put set those flags in my ~/.R/Makevars-PLATFORM file, as it seems the build step is picking these up so I don't have to pass them into the configure.args param. For the record, since I'm on a first generation MBP, the name of that file is "Makevars-i386-apple-darwin8.11.1", and it looks like: CC=/usr/bin/gcc CPP=/usr/bin/cpp CXX=/usr/bin/g++ CXXCPP=/usr/bin/cpp OBJC=/usr/bin/gcc Kasper:
You will want to recompile R with the new compiler. Then, whenever you compile a package, it will use the same compiler as R was compiled with.
Thanks for the preemptive warning. On the computer where I'm having this problem, R is actually installed from the official cran installer, so I just needed to set it to use the apple gcc by default. I have to assume that this R was also built w/ Apple's gcc, so perhaps R doesn't use the same compiler by default, as you suggest? I'm not sure.
Mixing compilers might be possible using the hints from Duncan, but I am pretty sure it is discouraged. In the past Simon has discouraged use of the HPC compilers. I don't remember the reasons, but I respect the source of the information :) He knows way more about mac compilers than I do. Of course, this was a while back and things might have changed.
Yeah, I'd trust that source of information as well :-) This kind of leads me into another related question, then. So, I actually d/l'd the HPC compiler so I can compile w/ -fopenmp (to use OpenMP for some easy parallelization). Does this mean that I shouldn't do that w/ a vanilla R install and perhaps recompile R from source w/ the HPC compiler? And if Simon doesn't like using the HPC compiler, then should we stay away from this in general? Thanks, -steve -- Steve Lianoglou Graduate Student: Physiology, Biophysics and Systems Biology Weill Medical College of Cornell University http://cbio.mskcc.org/~lianos
Sorry to spam, but I just wanted to post this follow up just in case someone asks google about this later. I found that I can get the R's default settings for the ones I needed to override (CC, CPP, etc.) by typing `R CMD config SETTING` at the command line. So, for instance, to find out what R had config'd for CPP: $ R CMD config CPP gcc -arch i386 -std=gnu99 -E So I used that to redefine my Makevars, only now I have an absolute to gcc/g++ so that I can get the ones from Apple, and not the one in my / usr/local/bin. So, the contents of my Makevars file is now: CC=/usr/bin/gcc -arch i386 -std=gnu99 CPP=/usr/bin/gcc -arch i386 -std=gnu99 -E CXX=/usr/bin/g++ -arch i386 CXXCPP=/usr/bin/g++ -arch i386 -E OBJC=/usr/bin/gcc -arch i386 -std=gnu99 -steve
On May 5, 2009, at 3:03 PM, Steve Lianoglou wrote:
Thanks Duncan & Kasper, I've been able to squeeze out of my problem ... Duncan:
You can pass configure args with the configure.args parameter to install.packages. I'm not sure how many you need to set, but you'll likely need to set CC, CPP, CXX, F77, FC, OBJC, and maybe some of the associated flags. You can see the list using "R CMD config" in the console.
The R CMD config was the trick. I actually put set those flags in my ~/.R/Makevars-PLATFORM file, as it seems the build step is picking these up so I don't have to pass them into the configure.args param. For the record, since I'm on a first generation MBP, the name of that file is "Makevars-i386-apple- darwin8.11.1", and it looks like: CC=/usr/bin/gcc CPP=/usr/bin/cpp CXX=/usr/bin/g++ CXXCPP=/usr/bin/cpp OBJC=/usr/bin/gcc Kasper:
You will want to recompile R with the new compiler. Then, whenever you compile a package, it will use the same compiler as R was compiled with.
Thanks for the preemptive warning. On the computer where I'm having this problem, R is actually installed from the official cran installer, so I just needed to set it to use the apple gcc by default. I have to assume that this R was also built w/ Apple's gcc, so perhaps R doesn't use the same compiler by default, as you suggest? I'm not sure.
Mixing compilers might be possible using the hints from Duncan, but I am pretty sure it is discouraged. In the past Simon has discouraged use of the HPC compilers. I don't remember the reasons, but I respect the source of the information :) He knows way more about mac compilers than I do. Of course, this was a while back and things might have changed.
Yeah, I'd trust that source of information as well :-) This kind of leads me into another related question, then. So, I actually d/l'd the HPC compiler so I can compile w/ -fopenmp (to use OpenMP for some easy parallelization). Does this mean that I shouldn't do that w/ a vanilla R install and perhaps recompile R from source w/ the HPC compiler? And if Simon doesn't like using the HPC compiler, then should we stay away from this in general? Thanks, -steve -- Steve Lianoglou Graduate Student: Physiology, Biophysics and Systems Biology Weill Medical College of Cornell University http://cbio.mskcc.org/~lianos
-- Steve Lianoglou Graduate Student: Physiology, Biophysics and Systems Biology Weill Medical College of Cornell University http://cbio.mskcc.org/~lianos
Steve,
On May 5, 2009, at 9:03 PM, Steve Lianoglou wrote:
Thanks Duncan & Kasper, I've been able to squeeze out of my problem ... Duncan:
You can pass configure args with the configure.args parameter to install.packages. I'm not sure how many you need to set, but you'll likely need to set CC, CPP, CXX, F77, FC, OBJC, and maybe some of the associated flags. You can see the list using "R CMD config" in the console.
The R CMD config was the trick. I actually put set those flags in my ~/.R/Makevars-PLATFORM file, as it seems the build step is picking these up so I don't have to pass them into the configure.args param. For the record, since I'm on a first generation MBP, the name of that file is "Makevars-i386-apple- darwin8.11.1", and it looks like: CC=/usr/bin/gcc CPP=/usr/bin/cpp CXX=/usr/bin/g++ CXXCPP=/usr/bin/cpp OBJC=/usr/bin/gcc Kasper:
You will want to recompile R with the new compiler. Then, whenever you compile a package, it will use the same compiler as R was compiled with.
Thanks for the preemptive warning. On the computer where I'm having this problem, R is actually installed from the official cran installer, so I just needed to set it to use the apple gcc by default. I have to assume that this R was also built w/ Apple's gcc, so perhaps R doesn't use the same compiler by default, as you suggest? I'm not sure.
R uses the Apple compiler, but I'm not hard-coding the full path. In your case apparently your PATH has /usr/local/bin before the system paths, so it overrides the default. Another (simpler?) way to solve this is to simply move /usr/local/bin at the end of your PATH (recommended) and all should be well without changing any config files.
Mixing compilers might be possible using the hints from Duncan, but I am pretty sure it is discouraged. In the past Simon has discouraged use of the HPC compilers. I don't remember the reasons, but I respect the source of the information :) He knows way more about mac compilers than I do. Of course, this was a while back and things might have changed.
Yeah, I'd trust that source of information as well :-) This kind of leads me into another related question, then. So, I actually d/l'd the HPC compiler so I can compile w/ -fopenmp (to use OpenMP for some easy parallelization).
You can use -fopenmp with Apple's compilers as well, just make sure you make the gcc-4.2 compilers the default (sudo gcc_select 4.2 if you have gcc_spect on your system, otherwise change the symlinks like ln - sf gcc-4.2 /usr/bin/gcc etc. - or specify them directly as you did). Apple's gcc-4.2 compilers are supported by the CRAN binary.
Does this mean that I shouldn't do that w/ a vanilla R install and perhaps recompile R from source w/ the HPC compiler? And if Simon doesn't like using the HPC compiler, then should we stay away from this in general?
In general, yes. As you have seen yourself the HPC compilers are not complete, so they cannot build universal binaries. Also they don't use Apple's driver, so all Apple-specific flags will fail. I didn't check the last HPC release, but before they were partially broken causing miscompilations, included wrong binaries (i.e. even the compiler for Tiger would include symbols for Leopard so the binary would not work), etc. I was trying to contact the maintainer a few times, but without avail, so I don't think it's well supported, either. Cheers, Simon
Hi Simon,
R uses the Apple compiler, but I'm not hard-coding the full path. In your case apparently your PATH has /usr/local/bin before the system paths, so it overrides the default. Another (simpler?) way to solve this is to simply move /usr/local/bin at the end of your PATH (recommended) and all should be well without changing any config files.
I actually wanted to avoid doing this because I tend to install newer versions of standard things into /usr/local/* that I'd like to use instead of the default ones in /usr/bin/* so I was looking for a back door ... As for the rest of your comments (which I'll keep below for posterity), they're spot on. Thanks for the thorough and informative feedback. I've nuked all traces of the HPC compilers and will now just explicitly compile w/ /usr/bin/g++-4.2 when necessary ... that's a simple-enough sol'n for me until Apple decides to set some gcc >= 4.2 as the default compiler. Thanks again, -steve
This kind of leads me into another related question, then. So, I actually d/l'd the HPC compiler so I can compile w/ -fopenmp (to use OpenMP for some easy parallelization).
You can use -fopenmp with Apple's compilers as well, just make sure you make the gcc-4.2 compilers the default (sudo gcc_select 4.2 if you have gcc_spect on your system, otherwise change the symlinks like ln -sf gcc-4.2 /usr/bin/gcc etc. - or specify them directly as you did). Apple's gcc-4.2 compilers are supported by the CRAN binary.
Does this mean that I shouldn't do that w/ a vanilla R install and perhaps recompile R from source w/ the HPC compiler? And if Simon doesn't like using the HPC compiler, then should we stay away from this in general?
In general, yes. As you have seen yourself the HPC compilers are not complete, so they cannot build universal binaries. Also they don't use Apple's driver, so all Apple-specific flags will fail. I didn't check the last HPC release, but before they were partially broken causing miscompilations, included wrong binaries (i.e. even the compiler for Tiger would include symbols for Leopard so the binary would not work), etc. I was trying to contact the maintainer a few times, but without avail, so I don't think it's well supported, either. Cheers, Simon
-- Steve Lianoglou Graduate Student: Physiology, Biophysics and Systems Biology Weill Medical College of Cornell University http://cbio.mskcc.org/~lianos