Choosing Intel MKL BLAS when compiling R
Sorry, small update. I had this problem before when dealing with
Intel MKL, but I need to revisit it now -- I got the linking working
via alternatives, but this does not seem to work:
(from alternatives:)
Current `best' version is /usr/local/intel-11.1/mkl/lib/em64t/libmkl_core.so
Booting up R gives me this error:
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object
'/usr/apps/oa/lib64/R/library/stats/libs/stats.so':
/usr/apps/oa/lib64/R/library/stats/libs/stats.so: undefined symbol: dswap_
During startup - Warning message:
package 'stats' in options("defaultPackages") was not found
I had this problem before -- is libmkl_core.so not the right file to
be linking against? If so, which one should I be using? Here's a
dump of the em64t folder:
libmkl_avx.so libmkl_gf_lp64.a libmkl_pgi_thread.so
libmkl_blacs_ilp64.a libmkl_gf_lp64.so
libmkl_scalapack_ilp64.a
libmkl_blacs_intelmpi_ilp64.a libmkl_gnu_thread.a
libmkl_scalapack_ilp64.so
libmkl_blacs_intelmpi_ilp64.so libmkl_gnu_thread.so libmkl_scalapack_lp64.a
libmkl_blacs_intelmpi_lp64.a libmkl_intel_ilp64.a
libmkl_scalapack_lp64.so
libmkl_blacs_intelmpi_lp64.so libmkl_intel_ilp64.so libmkl_sequential.a
libmkl_blacs_lp64.a libmkl_intel_lp64.a libmkl_sequential.so
libmkl_blacs_openmpi_ilp64.a libmkl_intel_lp64.so libmkl_solver_ilp64.a
libmkl_blacs_openmpi_lp64.a libmkl_intel_sp2dp.a
libmkl_solver_ilp64_sequential.a
libmkl_blacs_sgimpt_ilp64.a libmkl_intel_sp2dp.so libmkl_solver_lp64.a
libmkl_blacs_sgimpt_lp64.a libmkl_intel_thread.a
libmkl_solver_lp64_sequential.a
libmkl_blas95_ilp64.a libmkl_intel_thread.so libmkl_vml_avx.so
libmkl_blas95_lp64.a libmkl_lapack.so libmkl_vml_def.so
libmkl_cdft_core.a libmkl_lapack95_ilp64.a libmkl_vml_mc.so
libmkl_core.a libmkl_lapack95_lp64.a libmkl_vml_mc2.so
libmkl_core.so libmkl_mc.so libmkl_vml_mc3.so
libmkl_def.so libmkl_mc3.so libmkl_vml_p4n.so
libmkl_gf_ilp64.a libmkl_p4n.so locale
libmkl_gf_ilp64.so libmkl_pgi_thread.a
--j
On Tue, Nov 1, 2011 at 3:40 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
Dirk: Thanks -- I was able to get this running on a Debian box previously (http://lostingeospace.blogspot.com/2010/07/multithreaded-r-using-blas-in-debian_28.html) but I'm having trouble figuring out specific steps I need to do to choose the Intel MKL BLAS, particularly now that I don't have a package manager to rely on -- is it an issue of using update-alternatives? ?How can I check to see what R is currently using? --j On Tue, Nov 1, 2011 at 3:12 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
On 1 November 2011 at 14:36, Jonathan Greenberg wrote: | r-sig-hpc'ers: | | I am compiling R 2.14 from scratch on a cluster computer (I can't use | any package managers for the install, unfortunately), and they have | the Intel MKL libraries available for BLAS/LAPACK. ?I have not | successfully gotten a configure/make/make install combo yet that uses | multiple CPUs to do, say: | | a = matrix(rnorm(5000*5000), 5000, 5000) | b = matrix(rnorm(5000*5000), 5000, 5000) | # This following line, minimally, should be getting distributed. | c = a%*%b | | First off, I've tried compiling it with the configure flags: | ./configure --prefix=/myprefix/ --enable-BLAS-shlib | | How do I then select the Intel MKL libraries to be used as the BLAS? | They are located in /usr/local/intel-11.1/mkl/lib/em64t BLAS is an interface so you do not have to configure for a given BLAS. You just build for the generic BLAS, and the drop better ones in. This is demonstrated in a little package / vignette / paper I have at ? https://r-forge.r-project.org/scm/viewvc.php/pkg/gcbd/?root=gcb and you may want to hit the pdf directly at ? https://r-forge.r-project.org/scm/viewvc.php/pkg/gcbd/inst/doc/gcbd.pdf?root=gcb&view=log This uses the package manager on Debian/Ubuntu to drop the basic Blas, Atlas, Goto and MKL in and out in a fully automatable (and hence scripted) manner so that benchmarks can be run. ? Besides the fact that is uses the dpkg package manager --- and of course the fact that the respective package maintainer have done the hard work of conforming to the Debian standard for pluggable Blas --- nothing is really distro specific. ?So you could use this as 'proof-of-concept' and try to make your cluster machines conform. In essence, this means trying to configure R to any external Blas and then plugging the MKL in. Some other details are in the 'R Installation and Administration' manual. Good luck, ?Dirk -- "Outside of a dog, a book is a man's best friend. Inside of a dog, it is too dark to read." -- Groucho Marx
-- Jonathan A. Greenberg, PhD Assistant Professor Department of Geography University of Illinois at Urbana-Champaign 607 South Mathews Avenue, MC 150 Urbana, IL 61801 Phone: 415-763-5476 AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007 http://www.geog.illinois.edu/people/JonathanGreenberg.html
Jonathan A. Greenberg, PhD Assistant Professor Department of Geography University of Illinois at Urbana-Champaign 607 South Mathews Avenue, MC 150 Urbana, IL 61801 Phone: 415-763-5476 AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007 http://www.geog.illinois.edu/people/JonathanGreenberg.html