An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20081105/b5d2a4c4/attachment.pl>
Building with MKL on Ubuntu
11 messages · Anand Patil, Martyn Plummer, Brian Ripley +1 more
It looks like the em64t version of MKL fails the test for the accuracy
of zdotu ("checking whether double complex BLAS can be used") and is
therefore dropped in favour of R's built-in BLAS. I have just tested
this on Fedora and get the same result.
The 32-bit MKL does work for me.
Martyn
On Wed, 2008-11-05 at 16:29 +0000, Anand Patil wrote:
On Tue, Nov 4, 2008 at 7:34 PM, <plummer at iarc.fr> wrote:
I can see a couple of problems: 1) No "-lpthread" in --with-blas 2) No "-L" prefix in the library path in --with-lapack In addition, I don't think you need to add -lmkl to --with-lapack, although that is probably harmless. Martyn Quoting Prof Brian Ripley <ripley at stats.ox.ac.uk>:
Look in config.log to see what's wrong. (E.g. is /opt/intel/mkl/10.0.2.018/lib/em64t in the ld.so cache?) And note the warnings in the manual about using --with-lapack: it is most definitely not recommended. R-devel would be a better place to ask questions about this.
Thanks Brian and Martyn, I've tried it again with two sets of configure options: ./configure --with-blas='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl -lguide -lpthread' --with-lapack='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl_lapack' --enable-R-shlib ./configure --with-blas='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl -lguide -lpthread' --enable-R-shlib In both cases, the result is the same: R builds and uses its own BLAS. I didn't find anything that looked like evidence that /opt/intel/mkl/ 10.0.2.018/lib/em64t was getting into the ld.so cache, ie I didn't find 'ld.so' and 'opt/intel...' close together anywhere. Summaries of the BLAS- or MKL-related bits in config.log follow, with the first set of options: configure:36563: checking for dgemm_ in -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread configure:36594: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread -lgfortran -lm -ldl -lm >&5 conftest.c: In function 'main': conftest.c:187: warning: implicit declaration of function 'dgemm_' configure:36600: $? = 0 configure:36616: result: yes configure:37408: checking whether double complex BLAS can be used configure:37481: result: no ... BLAS_LIBS0='' BLAS_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRblas' BLAS_SHLIB_FALSE='#' BLAS_SHLIB_TRUE='' ... JAVA_LD_LIBRARY_PATH='$(JAVA_HOME)/lib/amd64/server:$(JAVA_HOME)/lib/amd64:$(JAVA_HOME)/../lib/amd64:/usr/local/lib64::/usr/lib64:/lib64:/usr/local/lib64:/usr/lib:/usr/local/lib:/lib:/opt/intel/fce/10.1.018/lib:/opt/intel/ipp/ 5.3.4.080/em64t/sharedlib:/opt/intel/cce/10.1.018/lib:/opt/intel/mkl/10.0.2.018/lib/em64t:/usr/java/packages/lib/amd64:/lib:/usr/lib ' JAVA_LIBS0='-L$(JAVA_HOME)/lib/amd64/server -L$(JAVA_HOME)/lib/amd64 -L$(JAVA_HOME)/../lib/amd64 -L/usr/local/lib64 -L -L/usr/lib64 -L/lib64 -L/usr/local/lib64 -L/usr/lib -L/usr/local/lib -L/lib -L/opt/intel/fce/10.1.018/lib -L/opt/intel/ipp/5.3.4.080/em64t/sharedlib-L/opt/intel/cce/10.1.018/lib -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -L/usr/java/packages/lib/amd64 -L/lib -L/usr/lib -ljvm' LAPACK_LDFLAGS='' LAPACK_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRlapack' and with the second set: configure:36563: checking for dgemm_ in -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread configure:36594: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread -lgfortran -lm -ldl -lm >&5 conftest.c: In function 'main': conftest.c:187: warning: implicit declaration of function 'dgemm_' configure:36600: $? = 0 configure:36616: result: yes configure:37408: checking whether double complex BLAS can be used configure:37481: result: no ... BLAS_LIBS0='' BLAS_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRblas' BLAS_SHLIB_FALSE='#' BLAS_SHLIB_TRUE='' ... JAVA_LD_LIBRARY_PATH='$(JAVA_HOME)/lib/amd64/server:$(JAVA_HOME)/lib/amd64:$(JAVA_HOME)/../lib/amd64:/usr/local/lib64::/usr/lib64:/lib64:/usr/local/lib64:/usr/lib:/usr/local/lib:/lib:/opt/intel/fce/10.1.018/lib:/opt/intel/ipp/ 5.3.4.080/em64t/sharedlib:/opt/intel/cce/10.1.018/lib:/opt/intel/mkl/10.0.2.018/lib/em64t:/usr/java/packages/lib/amd64:/lib:/usr/lib ' JAVA_LIBS0='-L$(JAVA_HOME)/lib/amd64/server -L$(JAVA_HOME)/lib/amd64 -L$(JAVA_HOME)/../lib/amd64 -L/usr/local/lib64 -L -L/usr/lib64 -L/lib64 -L/usr/local/lib64 -L/usr/lib -L/usr/local/lib -L/lib -L/opt/intel/fce/10.1.018/lib -L/opt/intel/ipp/5.3.4.080/em64t/sharedlib-L/opt/intel/cce/10.1.018/lib -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -L/usr/java/packages/lib/amd64 -L/lib -L/usr/lib -ljvm' LAPACK_LDFLAGS='' LAPACK_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRlapack' Do either of these give us clues as to what's wrong? Thanks, Anand [[alternative HTML version deleted]]
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
-----------------------------------------------------------------------
This message and its attachments are strictly confidenti...{{dropped:8}}
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20081105/dde64a99/attachment.pl>
You can't link 32-bit libraries into 64-bit code. That test is not for accuracy but for compatible return conventions. What compilers are in use here? We've seen several instances of gcc and icc not being compatible on x86_64 (Linux and Mac OS X: not surprising as gcc3 and gcc4 are also not compatible in their return conventions), and presumably MKL is compiled with icc. Our experience with Intel compilers and MKL is uniformly negative -- less accuracy and usually slower runs.
On Wed, 5 Nov 2008, Anand Patil wrote:
On Wed, Nov 5, 2008 at 5:59 PM, Martyn Plummer <plummer at iarc.fr> wrote:
It looks like the em64t version of MKL fails the test for the accuracy
of zdotu ("checking whether double complex BLAS can be used") and is
therefore dropped in favour of R's built-in BLAS. I have just tested
this on Fedora and get the same result.
The 32-bit MKL does work for me.
Martyn
Many thanks, Martyn. The 64-bit index space is important to me, will I lose it if I link against the 32-bit MKL? Also, should I file a bug report with Intel? Anand [[alternative HTML version deleted]]
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Anand Patil wrote:
On Wed, Nov 5, 2008 at 5:59 PM, Martyn Plummer <plummer at iarc.fr> wrote:
It looks like the em64t version of MKL fails the test for the accuracy
of zdotu ("checking whether double complex BLAS can be used") and is
therefore dropped in favour of R's built-in BLAS. I have just tested
this on Fedora and get the same result.
The 32-bit MKL does work for me.
Martyn
Many thanks, Martyn. The 64-bit index space is important to me, will I lose it if I link against the 32-bit MKL? Also, should I file a bug report with Intel? Anand
There's also the option of "breaking the thermometer". You might examine
that check and decide whether the loss of accuracy is enough for you to
worry about and if not, take out the test from configure.
Apparently, this check was put in place in R-2.2.0
o Any external BLAS found is now tested to see if the complex
routine zdotu works correctly: this provides a compatibility
test of compiler return conventions.
which suggests that the expected failure is catastrophic, and looking at
the code, there's a fuzz of 1e-10 which would seem to be about 1e5 times
larger than required.
O__ ---- Peter Dalgaard ?ster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk) FAX: (+45) 35327907
Quoting Peter Dalgaard <P.Dalgaard at biostat.ku.dk>:
Anand Patil wrote:
On Wed, Nov 5, 2008 at 5:59 PM, Martyn Plummer <plummer at iarc.fr> wrote:
It looks like the em64t version of MKL fails the test for the accuracy
of zdotu ("checking whether double complex BLAS can be used") and is
therefore dropped in favour of R's built-in BLAS. I have just tested
this on Fedora and get the same result.
The 32-bit MKL does work for me.
Martyn
Many thanks, Martyn. The 64-bit index space is important to me, will I lose it if I link against the 32-bit MKL? Also, should I file a bug report with Intel? Anand
There's also the option of "breaking the thermometer". You might examine
that check and decide whether the loss of accuracy is enough for you to
worry about and if not, take out the test from configure.
Apparently, this check was put in place in R-2.2.0
o Any external BLAS found is now tested to see if the complex
routine zdotu works correctly: this provides a compatibility
test of compiler return conventions.
which suggests that the expected failure is catastrophic, and looking at
the code, there's a fuzz of 1e-10 which would seem to be about 1e5 times
larger than required.
My apologies. I skimmed the code of the test program, but as Brian says
it is not a question of accuracy. The test program does in fact segfault,
although you won't see this in your config.log.
Anyway, Anand, I would just carry on with libRblas.
Martyn
-----------------------------------------------------------------------
This message and its attachments are strictly confidenti...{{dropped:8}}
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20081105/e6033392/attachment.pl>
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20081105/7b7cc872/attachment.pl>
And how does that help you? At some future point on a real problem you will get incorrect answers/segfault. That test was put there as a result of such incorrect results found the hard way.
On Wed, 5 Nov 2008, Anand Patil wrote:
Scratch that- finally got it working! The steps were:
- Change configure to 'break the thermometer' as suggested:
===================================================================
--- configure ? (revision 46843)
+++ configure ? (working copy)
@@ -37473,6 +37473,8 @@
?
?fi
?
+r_cv_zdotu_is_usable=yes
+
?? rm -rf conftest conftest.* conftestf.* core
?? if test -n "${r_cv_zdotu_is_usable}"; then
?? ? { echo "$as_me:$LINENO: result: yes" >&5
- Set environment variables:
CFLAGS=-pthread -O3
FC=gfortran -pthread
FFLAGS=-pthread
CXXFLAGS=-O3 -pthread
FCLAGS=-pthread
LDFLAGS=-lpthread
- Configure command:
./configure --with-blas='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl -lguide
-lpthread' --enable-R-shlib
I owe you all a beer!
Anand
On Wed, Nov 5, 2008 at 9:10 PM, Anand Patil
<anand.prabhakar.patil at gmail.com> wrote:
Thanks for your help, everyone. I was using gcc previously, but
building with the Intel compilers causes its own problems:
/opt/intel/cce/10.1.018/lib/libguide.so: undefined reference to
`pthread_atfork'
make[3]: *** [R.bin] Error 1
make[3]: Leaving directory `/working_copies/R/src/main'
make[2]: *** [R] Error 2
make[2]: Leaving directory `/working_copies/R/src/main'
make[1]: *** [R] Error 1
make[1]: Leaving directory `/working_copies/R/src'
make: *** [R] Error 1
If your experience with MKL has been negative, I've definitely had
enough; I'll try my luck with gcc and GotoBLAS.?
Cheers,
Anand
On Wed, Nov 5, 2008 at 9:02 PM, <plummer at iarc.fr> wrote:
Quoting Peter Dalgaard <P.Dalgaard at biostat.ku.dk>:
> Anand Patil wrote:
> > On Wed, Nov 5, 2008 at 5:59 PM, Martyn Plummer
<plummer at iarc.fr> wrote:
> >
> >> It looks like the em64t version of MKL fails the test
for the accuracy
> >> of zdotu ("checking whether double complex BLAS can
be used") and is
> >> therefore dropped in favour of R's built-in BLAS. ?I
have just tested
> >> this on Fedora and get the same result.
> >>
> >> The 32-bit MKL does work for me.
> >>
> >> Martyn
> >>
> >
> > Many thanks, Martyn. The 64-bit index space is
important to me, will I lose
> > it if I link against the 32-bit MKL? Also, should I
file a bug report with
> > Intel?
> > Anand
>
> There's also the option of "breaking the thermometer".
You might examine
> that check and decide whether the loss of accuracy is
enough for you to
> worry about and if not, take out the test from
configure.
>
> Apparently, this check was put in place in R-2.2.0
>
> ? ? o Any external BLAS found is now tested to see if
the complex
> ? ? ? ? routine zdotu works correctly: this provides a
compatibility
> ? ? ? ? test of compiler return conventions.
>
> which suggests that the expected failure is
catastrophic, and looking at
> the code, there's a fuzz of 1e-10 which would seem to be
about 1e5 times
> larger than required.
My apologies. I skimmed the code of the test program, but as
Brian says
it is not a question of accuracy. The test program does in fact
segfault,
although you won't see this in your config.log.
Anyway, Anand, I would just carry on with libRblas.
Martyn
-----------------------------------------------------------------------
This message and its attachments are strictly confiden...{{dropped:26}}
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20081106/86c79f4d/attachment.pl>
On Thu, 6 Nov 2008, Anand Patil wrote:
On Thu, Nov 6, 2008 at 6:48 AM, Prof Brian Ripley <ripley at stats.ox.ac.uk> wrote:
And how does that help you? ?At some future point on a real problem you will get
incorrect answers/segfault.
That test was put there as a result of such incorrect results found the hard way.
I appreciate that... but I never use the complex blas, so in my case the test is preventing me
from taking advantage of MKL's real blas. Or is the zdotu error symptomatic of a problem that
would affect the real blas also?
Only complex BLASAFAIK, *but* e.g. lapack uses it internally, AFAIK even for real inputs.
Anand
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595