Skip to content

Build R with MKL and ICC

12 messages · Martyn Plummer, Arnaud Gaboury, Bjørn-Helge Mevik +1 more

Original
Arnaud Gaboury
# 
After a few days of reading and headache, I finally gave a try at
building R from source with Intel MKL and ICC. Documentation and posts
on this topic are rather incomplete, sometime fantasist et do not give
much explanations about configure options.
As I am not sure if mine is correct, I would appreciate some advices and hints.

OS: Fedora 22
parallel_studio_xe_2016
Hardware : 8 Thread(s) per core: 2 Vendor ID: GenuineIntel Model name:
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz - Sandybridge
R-3.2.2

Here is my build configuration:

-------------------------------------------------------
source /opt/intel/compilers_and_libraries_2016/linux/mkl/bin/mklvars.sh intel64
source /opt/intel/bin/compilervars.sh intel64
_mkllibpath=$MKLROOT/lib/intel64
_icclibpath=$MKLROOT/linux/compiler/lib
export LD_LIBRARY_PATH=${_mkllibpath}:${_icclibpath}
export MKL="-L${_mkllibpath} -L${_icclibpath} -lmkl_intel_lp64
-lmkl_intel_thread -lmkl_core  -liomp5 -lpthread"
export CC="icc"
export F77="ifort"
export CXX="icpc"
export AR="xiar"
export LD="xild"
export CFLAGS="-O3 -ipo -openmp -parallel -xAVX"
export CXXFLAGS="-O3 -ipo -openmp -parallel -xAVX"
export FFLAGS="-O3 -ipo -openmp -parallel -xAVX"
export MAIN_LDFLAGS='-openmp'
./configure --with-lapack --with-blas="$MKL" --enable-R-shlib
--enable-memory-profiling --enable-openmp --enable-BLAS-shlib
--enable-lto F77=${F77} FC=${F77}
------------------------------------------------------------

After I run ./configure, it seems from config.log everything is fine:

checking for dgemm_ in
result: yes

checking whether double complex BLAS can be used
result: yes

checking whether the BLAS is complete
result: yes

The only error I can see is ld complaining about not finding -lRblas
----------------------------------------------------------------------------

Then run $ make with no errors.
Now, with no $ make install, I get this:

--------------------------------------------------------------------
$ ldd bin/exec/R
linux-vdso.so.1 (0x00007ffe073f3000)
libR.so => /usr/lib64/R/lib/libR.so (0x00007f43939e6000)
libRblas.so => not found
libm.so.6 => /lib64/libm.so.6 (0x00007f43936de000)
libiomp5.so => /opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64/libiomp5.so
(0x00007f439339c000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f4393185000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4392f69000)
libc.so.6 => /lib64/libc.so.6 (0x00007f4392ba8000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f43929a4000)
libblas.so.3 => /lib64/libblas.so.3 (0x00007f439274b000)
libgfortran.so.3 => /lib64/libgfortran.so.3 (0x00007f439241f000)
libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007f43921e0000)
libreadline.so.6 => /lib64/libreadline.so.6 (0x00007f4391f96000)
libtre.so.5 => /lib64/libtre.so.5 (0x00007f4391d85000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f4391b15000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f43918ef000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f43916de000)
libz.so.1 => /lib64/libz.so.1 (0x00007f43914c8000)
librt.so.1 => /lib64/librt.so.1 (0x00007f43912c0000)
libicuuc.so.54 => /lib64/libicuuc.so.54 (0x00007f4390f2e000)
libicui18n.so.54 => /lib64/libicui18n.so.54 (0x00007f4390ad7000)
libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f43908b5000)
/lib64/ld-linux-x86-64.so.2 (0x00005557e2243000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f439068a000)
libicudata.so.54 => /lib64/libicudata.so.54 (0x00007f438ec5f000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f438e8dc000)
-----------------------------------------------------------------------------------------------


Now a few questions:

1- am I not supposed to see something like this in the ldd command return?
libmkl_intel_lp64.so => /opt/intel....
libmkl_intel_thread.so => /opt/intel/....
libmkl_core.so => /opt/intel/...

Or do I need to run $make install before ldd?

2- when visiting Intel MKL link advisor[0], here is what I get as
configure and make options:
Linking: -L${MKLROOT}/lib/intel64 -lmkl_intel_ilp64 -lmkl_core
-lmkl_intel_thread -lpthread -lm
Compiler options:  -DMKL_ILP64 -qopenmp -I${MKLROOT}/include

What is the difference between -openmp and -qopenmp? Shall I use
indeed the above compiler options?

Thank you for help in this difficult topic for me.
Arnaud Gaboury
#
On Wed, Sep 2, 2015 at 7:35 PM, arnaud gaboury <arnaud.gaboury at gmail.com> wrote:
EDIT

  
    

      
      
    

  


  


      

    

    
1 day later

    

      
      

        


  

        

      Bjørn-Helge Mevik
    

    

      
      #
    

  


  

      
arnaud gaboury <arnaud.gaboury at gmail.com> writes:

            

      
      
      
    

        
This is how we build R with the Intel compilers and MKL on CentOS 6.x,
with different versions of R (latest version: 3.2.1) and Intel compilers
(latest version: 2015.3) on Intel SandyBridge CPUs:

fast="-ip -O3 -opt-mem-layout-trans=3 -xHost -mavx"
export CC="icc"
export CFLAGS="$fast -wd188 -fp-model precise"
export F77="ifort"
export FFLAGS="$fast -fp-model precise"
export CXX="icpc"
export CXXFLAGS="$fast -fp-model precise"
export FC="ifort"
export FCFLAGS="$fast -fp-model precise"

./configure --prefix=$destdir --with-blas='-mkl=parallel' --with-lapack --enable-BLAS-shlib


## Notes (they might not be true anymore):
## -static and -ipo break compilation
## -no-prec-div breaks make check
## -fp-model precise is needed for make check
## -wd188 removes a lot of warnings (see R Inst. & Adm. manual)

We have successfully used this setup for a couple of years now.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Arnaud Gaboury
    

    

      
      #
    

  


  

      
On Fri, Sep 4, 2015, 9:24 AM Bj?rn-Helge Mevik <b.h.mevik at usit.uio.no>
wrote:

arnaud gaboury <arnaud.gaboury at gmail.com> writes:

            

      
      
      
    

        
hints.

This is how we build R with the Intel compilers and MKL on CentOS 6.x,
with different versions of R (latest version: 3.2.1) and Intel compilers
(latest version: 2015.3) on Intel SandyBridge CPUs:

fast="-ip -O3 -opt-mem-layout-trans=3 -xHost -mavx"
export CC="icc"
export CFLAGS="$fast -wd188 -fp-model precise"
export F77="ifort"
export FFLAGS="$fast -fp-model precise"
export CXX="icpc"
export CXXFLAGS="$fast -fp-model precise"
export FC="ifort"
export FCFLAGS="$fast -fp-model precise"

./configure --prefix=$destdir --with-blas='-mkl=parallel' --with-lapack
--enable-BLAS-shlib


Thank you for this configuration. Nothing about openmp?

What about my cited missing librairies.
libmkl_intel_lp64.so
libmkl_intel_thread.so
libmkl_core.so

Any idea why they are not listed by the ldd command?

## Notes (they might not be true anymore):
## -static and -ipo break compilation
## -no-prec-div breaks make check
## -fp-model precise is needed for make check
## -wd188 removes a lot of warnings (see R Inst. & Adm. manual)

We have successfully used this setup for a couple of years now.

--
Regards,
Bj?rn-Helge Mevik

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

  
  


  


      

    

    

      
      

        


  

        

      Martyn Plummer
    

    

      
      #
    

  


  

      
On Wed, 2015-09-02 at 20:49 +0200, arnaud gaboury wrote:

            

      
      
      
    

        
You are looking in the wrong place. With the option --enable-BLAS-shlib,
R is linked to MKL via the library libRblas.so which you will find in
the directory "lib" after building R.

            

      
      
      
    

        
The option -qopenmp replaces -openmp, which is deprecated. This is in
the man page for icc.

Martyn

            

      
      
      
    

        
-----------------------------------------------------------------------
This message and its attachments are strictly confidenti...{{dropped:8}}

  
  


  


      

    

    

      
      

        


  

        

      Arnaud Gaboury
    

    

      
      #
    

  


  

      
On Fri, Sep 4, 2015 at 5:58 PM, Martyn Plummer <plummerm at iarc.fr> wrote:

            

      
      
      
    

        
Right.
$ ls lib
libRblas.so*  libRlapack.so*  libR.so*

$ ldd bin/exec/R returns
....
libRblas.so => not found

Shall I let things like that or tell R the path to libRblas.so ?

            

      
      
      
    

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      Bjørn-Helge Mevik
    

    

      
      #
    

  


  

      
arnaud gaboury <arnaud.gaboury at gmail.com> writes:

            

      
      
      
    

        
I believe -mkl=parallel makes sure that all relevant libraries and
switches are used.  But you might want to check with the manual.  All I
can say is that it works.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martyn Plummer
    

    

      
      #
    

  


  

      
On Sat, 2015-09-05 at 11:53 +0200, arnaud gaboury wrote:

            

      
      
      
    

        
No that is not a problem. R knows where to find these libaries even if
they are not on the linker path. 

Your only problem is to tell R where the Intel libraries are at runtime,
typically by setting/modifying LD_LIBRARY_PATH before launching R, e.g.
on my desktop I use 

export LD_LIBRARY_PATH=/opt/intel/composerxe/lib/intel64:/opt/intel/composerxe/mkl/lib/intel64/

Martyn

            

      
      
      
    

        
-----------------------------------------------------------------------
This message and its attachments are strictly confidenti...{{dropped:8}}

  
  


  


      

    

    
2 days later

    

      
      

        


  

        

      Nathan Kurz
    

    

      
      #
    

  


  

      
As a short and simple approach, I just compiled the current R release
on Ubuntu with ICC and MKL using just this:

$ tar -xzf R-3.2.2.tar.gz
$ cd R-3.2.2
$ CC=icc CXX=icpc AR=xiar LD=xild CFLAGS="-g -O3 -xHost" CXXFLAGS="-g
-O3 -xHost" ./configure --with-blas="-lmkl_rt -lpthread" --with-lapack
--enable-memory-profiling --enable-R-shlib
$ make
$ sudo make install
$ R --version
R version 3.2.2 (2015-08-14) -- "Fire Safety"

If you have 'ifort' available, you would probably want to add it to
the list of environment variables.

--nate

  
  


  


      

    

    
19 days later

    

      
      

        


  

        

      Arnaud Gaboury
    

    

      
      #
    

  


  

      
On Wed, Sep 9, 2015 at 11:26 PM, Nathan Kurz <nate at verse.com> wrote:

            

      
      
      
    

        
That is exactly the right combo: with-blas="-lmkl_rt -lpthread"
Nothing more for $MKL

now
$ ldd bin/exec/R
linux-vdso.so.1 (0x00007ffe305f9000)
libmkl_rt.so => /opt/intel/mkl/lib/intel64_lin/libmkl_rt.so (0x00007f216c9e3000)
        .................................

Thank you

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Nathan Kurz
    

    

      
      #
    

  


  

      
Hi Arnaud --

I'm glad it's working for you.  I'm not sure I understand your final
answer.  Are you saying that the version I posted worked for you as
given, or that you had to remove some of the other options?

Perhaps you could post the full final recipe in a way that others can
copy and paste if they find this thread in the future?

Also, did you determine if the the same approach worked for linking to
MKL worked when using a non-Intel compiler?   That is, can you
substitute the gcc or clang tool names and get the same result?

--nate

On Tue, Sep 29, 2015 at 9:07 AM, arnaud gaboury

        
<arnaud.gaboury at gmail.com> wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Arnaud Gaboury
    

    

      
      #
    

  


  

      
On Tue, Sep 29, 2015 at 8:11 PM, Nathan Kurz <nate at verse.com> wrote:

            

      
      
      
    

        
I say it works perfectly when using the single dynamic library
(lmkl_rt):  MKL=" -L${_mkllibpath} -lmkl_rt -lpthread -lm"
N.B: not sure the -lm is needed, and It builds without it. It seems it
may trigger some issues on. See[0]. Thus I removed it

            

      
      
      
    

        
Here it is. The script is writen for Parallel studio 2016. Change path
accordingly if composerxe 2015.
libRlib and libRblas are build as shared librairies. It build with
static libraries too.
More options can be added to configure.
Best is to source mklvars and compilervars in a script so it won't
polluated your shell environment once the build is done.

---------------------------------------------------------------
#! /bin/sh

source /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/bin/mklvars.sh
intel64
source /opt/intel/bin/compilervars.sh intel64

_icclibpath=/opt/intel/compilers_and_lbraries_2016.0.109/linux/compiler/lib/intel64/
_mkllibpath=${MKLROOT}/lib/intel64/
_omp_lib=/opt/intel/lib/intel64/

MKL=" -L${_mkllibpath} -lmkl_rt -lpthread"

export CC="icc"
export CXX="icpc"
export AR="xiar"
export LD="xild"
export F77="ifort"

export CFLAGS="-g -O3 -xHost -I${MKLROOT}/include"
export CXXFLAGS="-g -O3 -xHost -I${MKLROOT}/include"
export FFLAGS="-I${MKLROOT}/include"
export FCFLAGS="-I${MKLROOT}/include"

# shared libs
./configure --with-blas="${MKL}" --enable-R-shlib --enable-BLAS-shlib
# static libs
./configure --with-blas="${MKL}" --with-lapack

make -j4
----------------------------------------------------------------------------------------------

from inside the build folder:
-------------------------
$ldd bin/exec/R
linux-vdso.so.1 (0x00007ffc1e853000)
libR.so => /usr/lib64/R/lib/libR.so (0x00007f747091f000)
libRblas.so => not found         !! is not yet installed in the machine !!
libm.so.6 => /lib64/libm.so.6 (0x00007f7470616000)
libiomp5.so => /opt/intel/lib/intel64_lin/libiomp5.so (0x00007f74702d5000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f74700be000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f746fea1000)
libc.so.6 => /lib64/libc.so.6 (0x00007f746fae1000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f746f8dd000)
libblas.so.3 => not found         !! we don't need it !!
libgfortran.so.3 => /lib64/libgfortran.so.3 (0x00007f746f5b1000)
libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007f746f372000)
libreadline.so.6 => /lib64/libreadline.so.6 (0x00007f746f127000)
libtre.so.5 => /lib64/libtre.so.5 (0x00007f746ef17000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f746eca7000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f746ea80000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f746e870000)
libz.so.1 => /lib64/libz.so.1 (0x00007f746e65a000)
librt.so.1 => /lib64/librt.so.1 (0x00007f746e451000)
libicuuc.so.54 => /lib64/libicuuc.so.54 (0x00007f746e0c0000)
libicui18n.so.54 => /lib64/libicui18n.so.54 (0x00007f746dc69000)
libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f746da46000)
/lib64/ld-linux-x86-64.so.2 (0x000055fcb8058000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f746d81c000)
libicudata.so.54 => /lib64/libicudata.so.54 (0x00007f746bdf0000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f746ba6e000)

$ ldd lib/libRblas.so
linux-vdso.so.1 (0x00007ffc5f3da000)
libmkl_rt.so => /opt/intel/mkl/lib/intel64_lin/libmkl_rt.so (0x00007f232203e000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2321e21000)
libm.so.6 => /lib64/libm.so.6 (0x00007f2321b19000)
libimf.so => /opt/intel/lib/intel64_lin/libimf.so (0x00007f2321621000)
libsvml.so => /opt/intel/lib/intel64_lin/libsvml.so (0x00007f2320762000)
libirng.so => /opt/intel/lib/intel64_lin/libirng.so (0x00007f2320559000)
libiomp5.so => /opt/intel/lib/intel64_lin/libiomp5.so (0x00007f2320218000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2320001000)
libintlc.so.5 => /opt/intel/lib/intel64_lin/libintlc.so.5 (0x00007f231fda1000)
libc.so.6 => /lib64/libc.so.6 (0x00007f231f9e1000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f231f7dd000)
/lib64/ld-linux-x86-64.so.2 (0x000055ce12e09000)
--------------------------------------------------------------------------

            

      
      
      
    

        
No. I was not enable to do it with gcc. More tries are needed with
some variations.

            

      
      
      
    

        
[0]https://software.intel.com/en-us/search/site/language/en?query=-lm