Skip to content

Building R 4.0.2 from source via clang/xcode for MKL on macOS

8 messages · rollin, Kasper Daniel Hansen, Brian Ripley +2 more

Original
rollin
# 
I wanted to build R from source on macos (10.15.5) so I could include
Intel's MKL.  So I first looked at building R from source without MKL.
CC=clang
OBJC=$CC
FC=/usr/local/bin/gfortran
CXX=clang++


I then ran configuration via the command:

./configure -C --enable-R-shlib --enable-memory-profiling
--x-includes=/opt/X11/include --x-libraries=/opt/X11/lib
PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig


And received the following information and error:

checking if bzip2 version >= 1.0.6... no
checking whether bzip2 support suffices... configure: error: bzip2 library
and headers are required


By looking at the log, I saw a compiler error due to an implicit function.
I then made the following change in config.site:

CFLAGS='-Wno-implicit-function-declaration -g -O2''


And configure now ran without errors.

However, when I looked at configuring to use MKL, I discovered that MKL on
macos does not support gnu fortran so, unless I purchase Intel's Fortran
compiler, it looks like I'm sol.

Has anyone built R with MKL on macos (10.15)?  In any event, I wanted to at
least note the issue and work around I encountered when building R on
macos with clang/xcode.
Simon Urbanek
# 
Rollin,

it has been several years since I last tested MKL, so take it with a grain of salt, but in general you don't necessarily have to build R with MKL in order to use it - you only need to use --enable-BLAS-shlib and link to any ABI-compatible BLAS which can be vecLib as well. Then you can change the link from vecLib to MKL in the BLAS stub. Note that we only need the C ABI, there are wrappers vecLibg95f.* which re-map the F entry points to C entry points as to avoid Fortran ABI issues thus you don't care about the Fortran. However, historically, MKL has not been much more performant than vecLib so it's unclear if it is worth the hassle. As with any accelerated BLAS, note that this may have effects on results in R.

Cheers,
Simon
roy
# 
Hi Simon,
Thanks for the info.  I was totally unaware of ABI, vecLib, etc and that
Apple has blas, lapack, etc.  But after reading up on this and re-reading
your email, I'm beginning to understand more about this.

So, I would like to first checkout vecLib.  From what you say, would I have
to do something like the following?

./configure --enable-BLAS-shlib --with-blas="-lBLAS"  ...

Is this also possible with LAPACK?

tx again.
cheers, roy


On Mon, Sep 28, 2020 at 3:01 PM Simon Urbanek <simon.urbanek at r-project.org>
wrote:

  
  


  


      

    

    

      
      

        


  

        

      Kasper Daniel Hansen
    

    

      
      #
    

  


  

      
In my understanding, almost all of the speed up comes from using an
optimized BLAS. Optimizing LAPACK tend to have negligible impact on
performance. At least that is my impression, although I won't say I'm an
expert.

Best,
Kasper

        
On Tue, Sep 29, 2020 at 9:53 AM roy <rollinforall at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Kasper Daniel Hansen
    

    

      
      #
    

  


  

      
To use veclib you need
   --with-blas="-framework Accelerate"

My full building script on OS X is currently

../${SRCDIR}/configure SHELL='/bin/bash' \
  --prefix=/usr/local/R/${R_VERSION} --disable-R-framework\
  CC="/usr/local/clang8/bin/clang" \
  CXX="/usr/local/clang8/bin/clang++" \
  F77="/usr/local/gfortran/bin/gfortran" \
  FC="/usr/local/gfortran/bin/gfortran" \
  OBJC="clang" \
  CFLAGS="-Wall -mtune=native -g -O2" \
  CXXFLAGS="-Wall -mtune=native -g -O2" \
  OBJCFLAGS="-Wall -mtune=native -g -O2 -fobjc-exceptions" \
  F77FLAGS="-Wall -g -O2 -mtune=generic" \
  FCFLAGS="$F77FLAGS" \
  FLIBS="-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0
-L/usr/local/gfortran/lib -lgfortran -lquadmath -lm" \
  LDFLAGS="-L/usr/local/clang8/lib -L/usr/local/lib" \
  DYLD_FALLBACK_LIBRARY_PATH="/usr/local/clang8/lib:/usr/local/lib" \

PKG_CONFIG_PATH="/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig"
\
  JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Home" \
  JAVA_CPPFLAGS="-I/System/Library/Frameworks/JavaVM.framework/Headers" \
  JAVA_LD_LIBRARY_PATH="" \
  JAVA_LIBS="-framework JavaVM" \
  --enable-memory-profiling\
  --x-includes=/opt/X11/include --x-libraries=/opt/X11/lib\
  --with-blas="-framework Accelerate" | tee ../configure-${R_VERSION}

although I think we have moved back to Apple's clang for C etc compiler for
the CRAN version of R instead of Simon's clang8.

Best,
Kasper

        
On Tue, Sep 29, 2020 at 9:53 AM roy <rollinforall at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
On 29/09/2020 12:27, Kasper Daniel Hansen wrote:

            

      
      
      
    

        
Details are in the R-admin manual, including that R fails one of its checks.

Simon's calling this vecLib is a bit misleading.  'vecLib' was a 
vectorized library for PPC Macs, and the name lives on for a small part 
of the Accelerate framework.  The part that supplies an enhanced BLAS is 
different, and is based on ATLAS for x86_64 CPUs.

Incidentally, on the configuration issue -- had you followed the posting 
guide and either read the current manual or tried R-patched you would 
not have encountered the problem (which AFAWK is very recent, from the 
Command Line Tools for Xcode 12, released only for rather recent 
versions of Catalina and pre-Big Sur about 10 days ago).  Now documented at

https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

          

      
      
      
    

        
This is just the way Apple names it (now on all platforms), the name has a long history and was related to PPC, but Apple chose not to change it even though it doesn't actually contain any PowerPC code anymore. Accelerate is just a stub that combines vecLib and vImage, and since R only uses vecLib it is sufficient to link to vecLib. I'm just referring to the actual framework involved and the vecLib framework itself contains all the libraries including LAPACK and BLAS ( https://developer.apple.com/documentation/accelerate/veclib ).

Cheers,
Simon

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      roy
    

    

      
      #
    

  


  

      
Thanks to all for the info and to Kasper for the complete options in
building R on macos.  Using xcode clang/blas, I successfully built R 4.0.2
with the accelerate framework (or vecLib).  I did some initial performance
tests  using the scripts from
https://blog.revolutionanalytics.com/2014/10/revolution-r-open-mkl.html (a
cross product and a Cholesky factorization).

   1. R 4.0.2 macos built with xcode/clang; without accelerate:    cross
   product: 151 secs elapsed / 151 secs user; factorization: 23 secs elapsed /
   23 secs user
   2. R 4.0.2 macos built with xcode/clang; with accelerate:        cross
   product: 1.7 secs elapsed / 6.3 secs user; factorization: 0.6 secs elapsed
   / 1.9 secs user
   3. R 4.0.2 w/ MKL sequential via ubuntu/docker on mac:        cross
   product: 5 secs elapsed / 5 secs user; factorization: 0.94 secs elapsed /
   0.9 secs user
   4. R 4.0.2 w/ MKL parallel via ubuntu/docker on mac:             cross
   product: 2.8 secs elapsed / 5.3 secs user; factorization: 0.55 secs elapsed
   / 0.9 secs user

btw, these tests were done on:

   - macos 10.15.7
   - 2018 mac mini
   - 3.6 GHZ, quad core, 16 GB memory


cheers, roy


On Wed, Sep 30, 2020 at 12:10 AM Simon Urbanek <simon.urbanek at r-project.org>
wrote: