Skip to content

Building R-devel from Source on Snow Leopard

6 messages · Adam D. I. Kramer, Simon Urbanek

Original
Adam D. I. Kramer
# 
Hi Tony and Simon,

 	I was having this trouble today/yesterday and have been following
along on this thread. The solution I found was to add this to my
./configure:

LDFLAGS="-m64 -L/usr/lib/gcc/i686-apple-darwin9/4.2.1/x86_64 -lgfortran"
and
F77="gfortran -arch x86_64"

...this gets me through the "can't make mixed C/Fortran code" errors. However, 
through one hoop of fire and I'm confronted by another:

checking for iconv.h... yes
checking for iconv... no
checking for iconvlist... no
configure: error: --with-iconv=yes (default) and a suitable iconv is not
available

...this strikes me as odd. The relevant lines from the config.log file are:

configure:39212: gcc -std=gnu99 -o conftest -O3 -fopenmp -mtune=native -m64  -m64 -L/usr/lib/gcc/i686-apple-darwin9/4.2.1/x86_64 -lgfortran conftest.c -lm >&5
conftest.c: In function 'main':
conftest.c:189: warning: passing argument 1 of 'libiconvlist' from
incompatible pointer type
Undefined symbols:
   "_libiconvlist", referenced from:
       _main in cc2XgkAg.o
ld: symbol(s) not found
collect2: ld returned 1 exit status

However:

swiss:R-2.9.2$ nm /usr/lib/libiconv.dylib | grep _libiconvlist
00008807 T _libiconvlist

...I'm not sure where else configure could be trying to find libiconv, but
running "locate libiconv" shows versions in
/Developer/SDKs/MacOSX10.6.sdk/usr/lib/ also, which also show _libiconvlist
when I nm them.

Any suggestions? I'm not sure how to further debug this...the symbols are
there.

Cordially,
Adam
Simon Urbanek
# 
Adam,
On Sep 4, 2009, at 5:52 PM, Adam D. I. Kramer wrote:

            

      
      
      
    

        
You should not touch LDFLAGS and besides F77 you'll also need to set FC.

            

      
      
      
    

        
check /usr/local/lib that is the most common problem area ...

Cheers,
Simon

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Adam D. I. Kramer
    

    

      
      #
    

  


  

      
Simon,

 	Thanks for your comments; responses below.

        
On Fri, 4 Sep 2009, Simon Urbanek wrote:

        

            

      
      
      
    

        
LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
               nonstandard directory <lib dir>

...the gfortran library lives in a nonstandard directory, which is why I put
the -L there... so I then moved -lgfortran to LIBS='-lgfortran' and things
broke in the same way, as did moving the -L...-lgfortran there.  Both of
these need to be passed to the linker when mixing C/Fortran code.

It also throws the error "Maybe check LDFLAGS for paths to Fortran
libraries?" which is what tipped me off to the fact that I should provide a
link to the Fortran libraries in LDFLAGS.  If this is incorrect, please let
me know where to put them instead (and consider updating the configure
script).

            

      
      
      
    

        
I had, and there was no libiconv there.

I eventually solved this problem by upgrading fink to the 64bit version and
installing libiconv and adding CPPFLAGS="-I/sw/include" and adding -L/sw/lib
to LDFLAGS, forcing it to link against the new library. It seems like the
Apple libraries should serve the purpose, but for whatever reason they did
not.

My current problem is that, during make, R complains about the lapack.so
file it has compiled from my lapack libraries (I configured with
--with-lapack="-L/usr/local -llapack -lptf77blas -lcblas -lpthread -latlas"
and a similar --with-blas):

Warning in solve.default(rgb) :
   unable to load shared library '/Volumes/Tubby2/nobackup/R-2.9.2/modules//lapack.so':
   dlopen(/Volumes/Tubby2/nobackup/R-2.9.2/modules//lapack.so, 6): Symbol not found: _cblas_cdotc_sub
   Referenced from: /Volumes/Tubby2/nobackup/R-2.9.2/modules//lapack.so
   Expected in: flat namespace
  in /Volumes/Tubby2/nobackup/R-2.9.2/modules//lapack.so
Error in solve.default(rgb) : lapack routines cannot be loaded
Error: unable to load R code in package 'grDevices'
Execution halted

...that said, I understand that --with-lapack is "not recommended" in the
admin manual, so I'm pretty much on my own here, but I'd be interested if
you have an idea or two about what might be going on. _cblas_cdotc_sub is
indeed in my lapack.a file (but not the lapack.so file R refers to), and I
can use _cblas_cdotc if I write some simple c code and link it against
lapack using the same flags as noted above.

My interest, really, is in whether ATLAS's LAPACK-tuning system (which is
pretty new) actually leads to speed improvements in R.  So, I don't
desperately need to use it, but am still concerned that a compile that used
to run smoothly no longer does.

--Adam

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Adam D. I. Kramer
    

    

      
      #
    

  


  

      
Hi Simon, Tony, and anybode else who attempts to complie 2.9.2 for
themselves on Snow Leopard,

The configure line which eventually led to R configuring, building, and
running was this:

./configure --prefix=/usr/local --with-x --disable-R-framework --enable-threads=posix --without-aqua --with-lapack=" /usr/local/lib/liblapack.a /usr/local/lib/libptf77blas.a /usr/local/lib/libptcblas.a -lpthread /usr/local/lib/libatlas.a " --with-blas=" /usr/local/lib/liblapack.a /usr/local/lib/libptf77blas.a /usr/local/lib/libptcblas.a -lpthread /usr/local/lib/libatlas.a " CFLAGS="-O3 -fopenmp -mtune=native -m64" CXXFLAGS="-O3 -fopenmp -mtune=native -m64" FFLAGS="-O3 -fopenmp -mtune=native -m64" LDFLAGS="-m64 -L/sw/lib -L/usr/X11R6/lib -L/usr/lib/gcc/i686-apple-darwin9/4.2.1/x86_64 -lgfortran" F77="gfortran -arch x86_64" CPPFLAGS="-I/sw/include" FC="gfortran -arch x86_64" FCFLAGS="-O3 -fopenmp -mtune=native -m64"

...dropping the --with-blas and --with-lapack items also lets R compile fine
(but see benchmarks below). Note also that I have libreadline and libiconv
installed in the /sw tree using 64-bit Fink for Snow Leopard...you'll need
64bit versions of readline and iconv to compile R; check out
/Developer/SDKs/MacOSX10.6.sdk/usr/lib maybe.

The one oddity is that when I pass

--with-blas="-L/usr/local/lib -llapack -lptf77blas -lptcblas -lpthread -latlas"
(and --with-lapack the same, or one and not the other)

...the build fails in the manner described in my last message (lapack.so,
compiled by R, ends up not containing the functions it should), while if I
explicitly list the libraries:

--with-blas=" /usr/local/lib/liblapack.a /usr/local/lib/libptf77blas.a /usr/local/lib/libptcblas.a -lpthread /usr/local/lib/libatlas.a

...it builds successfully. Not sure what this portends.

Also, I have done some benchmarking to see how much faster R is (on my Mac
Pro, dual quad-core Xeon @ 2.8GHz, 6G RAM) when I go through all this
trouble:

(benchmarked using http://r.research.att.com/benchmarks/R-benchmark-25.R)

nightly build
(http://r.research.att.com/R-2.9-branch-leopard-universal.tar.gz, built Sept
5 2009, run with --arch x86_64):
Overall mean (sum of I, II and III trimmed means/3)_ (sec):  0.835276859437347

my build, no BLAS or LAPACK:
Overall mean (sum of I, II and III trimmed means/3)_ (sec):  1.83643467491601 
(This build did consistently better on non-matrix math, FAR worse on the matrix math)

my build, BLAS, no LAPACK:
Overall mean (sum of I, II and III trimmed means/3)_ (sec):  0.760949143043198 
(outperforms nightly on most metrics, underperforms on none, outperforms no-blas on all)

my build, LAPACK & BLAS:
Overall mean (sum of I, II and III trimmed means/3)_ (sec):  0.70660451743888 
(outperforms everything else on most metrics, underperforms on none)

So it seems that letting ATLAS compile your BLAS and LAPACK (Atlas 3.9.14
does this), plus adding optimization flags, leads to a 15% improvement over
the nightly build (when using this specific benchmarking system, ymmv
depending on what functions you actually use R for).

--Adam

        
On Fri, 4 Sep 2009, Adam D. I. Kramer wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

      
On Sep 4, 2009, at 11:27 PM, Adam D. I. Kramer wrote:

        

            

      
      
      
    

        
Not really - it's the compiler library directory, i.e. where the  
compiler looks for its internal libraries. It is never used  
explicitly. The problem is that you're mixing compilers - you have a  
Leopard gfortran (see my response to John - you have the same problem)  
and SL gcc. That is not recommended and is likely to break in more  
places.

            

      
      
      
    

        
You should get a working gfortran instead - this is a problem in your  
setup, not configure. Unfortunately Apple has broken fortran in the  
gcc-4.2 tree so I don't have a working gfortran binary for Xcode 3.2,  
so please use the gfortran 4.2.3 from CRAN.

            

      
      
      
    

        
Oh, fink? Ok, that explains it all :).

            

      
      
      
    

        
They do, but fink overrides the look up sequence since most of its  
stuff is incompatible with the system versions (which is why it causes  
such a havoc).

Cheers,
Simon

            

      
      
      
    

        

      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

      
On Sep 5, 2009, at 7:10 PM, Adam D. I. Kramer wrote:

        

            

      
      
      
    

        
I would strong suggest *against* what you're doing, your configuration  
is far from any normal SL system. You're clearly free to use it for  
yourself but is not something any R-SIG-Mac users should follow. The  
setup is incompatible with CRAN and anything we build. (There are many  
reasons - stating with disabling aqua, using 3rd party BLAS (which is  
not a good idea if you want speed since vecLib is highly optimized)  
and mixing in fink (again, a really bad idea for any normal user)).

I see that the most common mistake here is to use Leopard Xcode 3.1  
fortran which is NOT meant to be used with SL and Xcode 3.2. To if you  
upgrade from Leopard and Xcode 3.1, make sure you delete the Xcode 3.1  
fortran as Xcode 3.2 is NOT able to upgrade it. Use the gfortran 4.2.3  
from CRAN instead - it works just fine and no special flags are  
necessary for SL (except for the mandatory -arch).

Cheers,
Simon