Skip to content

Build optimized R : openblas, MKL, ATLAS

7 messages · Arnaud Gaboury, Dirk Eddelbuettel, Bjørn-Helge Mevik

Original
Arnaud Gaboury
# 
I want to build R optimized, with either MKL, OpenBLAS or ATLAS.

My OS: Fedora 22
Hardware: CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian
CPU(s): 8 Thread(s) per core: 2 Vendor ID: GenuineIntel Model name:
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz

I am a little confused when it comes to choose a method and would like
to hear your experiences. If I am right, I have 3 possibilities:
- OpenBLAS: opensource and free, but I came across some posts
describing seg faults issues and bugs. These posts are 2 years old and
I wonder if it is still the case.
- ATLAS: can't see any reason to not use it
- Intel MKL: this is part of Intel Parallel Studio and is a paid
software. Now, there is the MKL package distributed by
Revolutionanalytics, but I am not certain how this can be distributed
for free. Is there any kind of difference? In case of use of this
package, do I need to install RRO or can I just build R from GNU
against these libraries?

Thank you for advices.
1 day later
Bjørn-Helge Mevik
# 
arnaud gaboury <arnaud.gaboury at gmail.com> writes:
We regularly build the standard R against MKL, simply using

./configure --with-blas='-mkl=parallel' --with-lapack --enable-BLAS-shlib

(The --enable-BLAS-shlib is not strictly needed for building agains MKL.)


We also try to optimise the build by setting some environment variables
before running configure:

fast="-ip -O3 -opt-mem-layout-trans=3 -xHost -mavx"
export CC="icc"
export CFLAGS="$fast -wd188 -fp-model precise"
export F77="ifort"
export FFLAGS="$fast -fp-model precise"
export CXX="icpc"
export CXXFLAGS="$fast -fp-model precise"
export FC="ifort"
export FCFLAGS="$fast -fp-model precise"
Arnaud Gaboury
# 
On Mon, Aug 24, 2015 at 11:29 AM, Bj?rn-Helge Mevik
<b.h.mevik at usit.uio.no> wrote:
Do you use proprietary Intel MKL or open source package like OpenBLAS ?

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 24 August 2015 at 11:43, arnaud gaboury wrote:

        
| On Mon, Aug 24, 2015 at 11:29 AM, Bj?rn-Helge Mevik

        
| <b.h.mevik at usit.uio.no> wrote:

        
| > arnaud gaboury <arnaud.gaboury at gmail.com> writes:
| >
| >> - Intel MKL: this is part of Intel Parallel Studio and is a paid
| >> software. Now, there is the MKL package distributed by
| >> Revolutionanalytics, but I am not certain how this can be distributed
| >> for free. Is there any kind of difference? In case of use of this
| >> package, do I need to install RRO or can I just build R from GNU
| >> against these libraries?
| >
| > We regularly build the standard R against MKL, simply using
| 
| Do you use proprietary Intel MKL or open source package like OpenBLAS ?

You may be able to switch at will _after_ R has been built and installed.

That is something people simply cannot get their head around but lapack and
blas are an INTERFACE for which several implementations provide suitable
code.

That is the gist behind the package gcbd -- which is really a container for a
paper / vignette where I use(d) the fact that on a Debian-based system I do
have access to several of these (including the MKL) so that I can even script
benchmark runs for comparison _from R_ as it is just a (system) package
removal / installation which can be automated.

The paper never got finished as the gpu/cpu comparison aspect confuses
matters further --- but the basic idea is solid:  you can still switch
later, so benchmarking is possibly.  And desirable as these question pop up
all the time.

For Debian/Ubuntu, I ensure we built using --with-blas --with-lapack and
after that you just swap the system libraries providing them.  You should
only need rebuilds of R if your system fails to make these swappable.

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Arnaud Gaboury
    

    

      
      #
    

  


  

      
On Mon, Aug 24, 2015 at 1:42 PM, Dirk Eddelbuettel <edd at debian.org> wrote:

            

      
      
      
    

        
something like:
# update-alternatives --config ....
?
Reference to this blog[0]

            

      
      
      
    

        
[0]http://blog.nguyenvq.com/blog/2014/11/10/optimized-r-and-python-standard-blas-vs-atlas-vs-openblas-vs-mkl/

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 24 August 2015 at 13:46, arnaud gaboury wrote:

        
| On Mon, Aug 24, 2015 at 1:42 PM, Dirk Eddelbuettel <edd at debian.org> wrote:

        
| >
| > You may be able to switch at will _after_ R has been built and installed.
| 
| something like:
| # update-alternatives --config ....
| ?
| Reference to this blog[0]
| >
| [0]http://blog.nguyenvq.com/blog/2014/11/10/optimized-r-and-python-standard-blas-vs-atlas-vs-openblas-vs-mkl/

Yes; see my post predating this (still using GOTO not OpenBLAS) 

    http://dirk.eddelbuettel.com/blog/2010/09/15/

and of course the package and vignette which has much more detail

    https://cran.rstudio.com/web/packages/gcbd/index.html
    
    https://cran.rstudio.com/web/packages/gcbd/vignettes/gcbd.pdf

We have a dedicated list r-sig-hpc which may be more suitable for this
discussion than continuing on r-devel.  [1]

Dirk


[1] Your "what web framework" question was also pretty much off-target there.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Bjørn-Helge Mevik
    

    

      
      #
    

  


  

      
arnaud gaboury <arnaud.gaboury at gmail.com> writes:

            

      
      
      
    

        
As I wrote: MKL.