Skip to content

problem installing rjags package

10 messages · Brian Ripley, Martyn Plummer, Michael Braun +1 more

Original
Michael Braun
# 
Hi!

I am trying to install rjags on my Mac Pro running OSX 10.6.4, and R 2.11.1 (which I compiled myself so I could link to the Intel MKL blas, which I prefer to other options).   I was successfully able to install JAGS 2.1.0 through both the binary installer, and compiling from source (I get the same problem either way).

But I would like to use the rjags package. When I do 

install.packages("rjags")


I get the folliowing output:

* installing *source* package ?rjags? ...
checking for prefix by checking for jags... /usr/local/bin/jags
checking for g++... g++
checking for C++ compiler default output file name... a.out
checking whether the C++ compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C++ preprocessor... g++ -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking Console.h usability... yes
checking Console.h presence... yes
checking for Console.h... yes
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for jags_version in -ljags... yes
configure: creating ./config.status
config.status: creating src/Makevars
configure: creating ./config.status
config.status: creating src/Makevars
config.status: creating R/unix/zzz.R
** libs
*** arch - x86_64
g++ -arch x86_64 -I/Library/Frameworks/R.framework/Resources/include -I/Library/Frameworks/R.framework/Resources/include/x86_64 -I/usr/local/include/JAGS -I/opt/intel/Compiler/11.1/089/Frameworks/mkl/Headers -I/Install_Files/MKL_gfortran_interface/include/em64t/lp64    -fPIC  -m64 -msse4.2 -g -O2 -c jags.cc -o jags.o
g++ -arch x86_64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/opt/intel/Compiler/11.1/089/Frameworks/mkl/Libraries/em64t -L/Users/braunm/Install_Files/MKL_gfortran_interface/lib/em64t -o rjags.so jags.o -lgsl -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Users/braunm/.R/Library/rjags/libs/x86_64
** R
** data
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
Error : .onLoad failed in loadNamespace() for 'rjags', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared library '/Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so':
  dlopen(/Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so, 10): Symbol not found: _JAGS_NA
  Referenced from: /Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so
  Expected in: flat namespace
 in /Users/braunm/.R/Library/rjags/libs/x86_64/rjags.so
ERROR: installing package indices failed
* removing ?/Users/braunm/.R/Library/rjags?


However, when I go to look at the symbols in the jags library, I see

braunm4: /usr/local/lib $ nm libjags.dylib |grep JAGS
0000000000098650 S _JAGS_NA
0000000000110338 S _JAGS_NAN
0000000000110348 S _JAGS_NEGINF
0000000000110340 S _JAGS_POSINF
000000000000133c t __GLOBAL__I_JAGS_NA

and when I try

braunm4: /usr/local/lib $ nm libjags.a |grep JAGS
                 U _JAGS_NA
                 U _JAGS_NEGINF
                 U _JAGS_NA
                 U _JAGS_NA
0000000000000060 S _JAGS_NA
00000000000000b8 B _JAGS_NAN
00000000000000c8 B _JAGS_NEGINF
00000000000000c0 B _JAGS_POSINF
                 U _JAGS_NA
                 U _JAGS_NEGINF
                 U _JAGS_POSINF
                 U _JAGS_NEGINF
                 U _JAGS_POSINF
                 U _JAGS_NEGINF
                 U _JAGS_NA
                 U _JAGS_NEGINF
                 U _JAGS_NEGINF
                 U _JAGS_NA
                 U _JAGS_NEGINF
                 U _JAGS_POSINF


So the jags libraries (both static and dynamic) are in the standard place, and it looks like the symbols rjags.so needs are there as well.  So I'm not sure what the rjags installer is looking for.  Is there perhaps an environment variable that I am missing?

Here is my sessionInfo():
R version 2.11.1 (2010-05-31) 
x86_64-apple-darwin10.4.2 

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     grDevices datasets  graphics  utils     methods   base     

other attached packages:
 [1] xtable_1.5-6     doMC_1.2.1       multicore_0.1-3  foreach_1.3.0    codetools_0.2-2  iterators_1.0.3  coda_0.13-5      matrixcalc_1.0-1 lattice_0.18-8   reshape_0.8.3    plyr_1.2.1      
[12] mvtnorm_0.9-92  

loaded via a namespace (and not attached):
[1] grid_2.11.1  tcltk_2.11.1 tools_2.11.1


Thanks in advance for your help.

Michael








-------------------------------------------
Michael Braun
Homer A. Burnell (1928) Career Development Professor, 
	and Assistant Professor of Management Science (Marketing Group)
MIT Sloan School of Management
100 Main St.., E62-535
Cambridge, MA 02139
braunm at mit.edu
617-253-3436
Brian Ripley
# 
Note your link line has no reference to -ljags.  Mine is (for i386, 
x86_64 is later on in the output)

g++ -arch i386 -dynamiclib -Wl,-headerpad_max_install_names -undefined 
dynamic_lookup -single_module -multiply_defined suppress 
-L/usr/local/lib -o rjags.so jags.o -L/usr/local/lib -ljags 
-F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework 
-Wl,CoreFoundation

with no reference to -lgsl.  Also, have you perhaps excluded 
/usr/local/lib (the usual place to install JAGS) from the R library 
paths?

Have you contacted the maintainer?  Although not a Mac user he is very 
knowledgable, including about MKL.
On Sat, 9 Oct 2010, Michael Braun wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martyn Plummer
    

    

      
      #
    

  


  

      
Michael,

I do have some experience of using MKL with JAGS and left quite
detailed instructions, for Linux, in the installation manual (section 1.3.5).

In addition to the problems noted by Brian there seem to be anomalies
with your MKL configuration:
- You are compiling with GNU compilers but trying to use the Intel
  64-bit interface layer. This will not work.
- You are using the sequential MKL library but still linking to
  libpthread, which is redundant.

As noted in the manual, on Linux I have been unable to link JAGS with
the MKL using GNU compilers, except by building a static version, which
is obviously not helpful if you want to interface to R.

To be honest, BLAS/LAPACK performance is not a bottleneck for JAGS unless
perhaps, you have a very special problem.  All of this effort to link
to MKL will probably not pay off and binary packages of both JAGS and rjags
are available and which do actually work.

Martyn

Quoting Prof Brian Ripley <ripley at stats.ox.ac.uk>:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        
-----------------------------------------------------------------------
This message and its attachments are strictly confidenti...{{dropped:8}}

  
  


  


      

    

    

      
      

        


  

        

      Michael Braun
    

    

      
      #
    

  


  

      
Thanks for the suggestions.  I did go through the JAGS installation manual slowly and carefully, but that doesn't mean I didn't miss something.

I may not have been clear about what I did.  The JAGS that is installed is the pre-compiled binary (I wanted to be sure that how I was compiling JAGS wasn't causing the problems).  The compilation flags below are generated by R (I used gcc to compile R,  and linked R to the MKL BLAS, and that seems to have worked well).  Perhaps I need to change my Makevars file for rjags?  Because (I think) R only (seems to) use the Makevars in my ~/.R folder, it's hard to keep track of what's linking to what across multiple projects (hence, why the -lgsl is in there). 

In any event, what did seem to work is when I ran R as root, and then installed rjags.  I can now load the library even when not running R as root.  So perhaps there was some kind of permissions problem that was keeping R from accessing the jags library (which is in /usr/local/lib).  Is there a particular R library path variable that I should be looking at?  I do have /usr/local/lib as part of R_DYLIB_LIBRARY_PATH, DYLD_LIBRARY_PATH, and LD_LIBRARY_PATH.

        
On Oct 10, 2010, at 8:11 AM, <plummer at iarc.fr> <plummer at iarc.fr> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        
-------------------------------------------
Michael Braun
Homer A. Burnell (1928) Career Development Professor, 
	and Assistant Professor of Management Science (Marketing Group)
MIT Sloan School of Management
100 Main St.., E62-535
Cambridge, MA 02139
braunm at mit.edu
617-253-3436



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-mac/attachments/20101010/8ef4adcf/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1844 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-mac/attachments/20101010/8ef4adcf/attachment-0001.bin>

  
  


  


      

    

    

      
      

        


  

        

      Michael Braun
    

    

      
      #
    

  


  

      
Problem solved.  Just adding -ljags to my Makevars file did the trick.  Thanks.
On that note, is it possible to customize Makevars for different R libraries (e.g., one for compiling R, one for CRAN packages, one for each personal C module that I build for personal use, etc.).  The manual seems to suggest otherwise?

        
On Oct 10, 2010, at 8:11 AM, <plummer at iarc.fr> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

      
Michael,

        
On Oct 9, 2010, at 5:56 PM, Michael Braun wrote:

        

            

      
      
      
    

        
Just FWIW: CRAN R comes with shared BLAS option enabled and allows easy plug-in of MKL BLAS without the need to re-compile (just use the libRblas.dylib symlink). Note that MKL BLAS is often slower than ATLAS so if your main goal is speed I suggest using ATLAS (or vecLib unless you have Nehalem-type Mac Pro) - it is also easier to handle (both from technical and licensing point of view).

Cheers,
Simon

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Michael Braun
    

    

      
      #
    

  


  

      
Thanks, Simon:

I do have a Nehalem Mac Pro, and I've found MKL to be much faster than vecLib.  I saw in one of the manuals that one could point R to another BLAS.  But I wasn't sure what to do with MKL because of their "layering" approach to linking.  So if I point libRblas to something else, how to I tell the link which of the multiple libraries I need?

And since we are now straying away from the original topic about JAGS, let me know if you prefer that I start another thread.  I do try to follow the posting guide as best I can.

Michael

        
On Oct 12, 2010, at 9:10 AM, Simon Urbanek wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        
-------------------------------------------
Michael Braun
Homer A. Burnell (1928) Career Development Professor, 
	and Assistant Professor of Management Science (Marketing Group)
MIT Sloan School of Management
100 Main St.., E62-535
Cambridge, MA 02139
braunm at mit.edu
617-253-3436

  
  


  


      

    

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

      
On Oct 12, 2010, at 11:19 AM, Michael Braun wrote:

        

            

      
      
      
    

        
Yes, because vecLib has a serious bug and doesn't spawn threads on Nehalem, but ATLAS is faster than MKL on Nehalem Mac Pro  -- here are the tcrossprod timings (on 2.66GHz Mac Pro):

vecLib           6.43
ATLAS serial     4.80
MKL serial       4.30
MKL parallel     0.90
ATLAS parallel   0.71

            

      
      
      
    

        
MKL is picking them at run time so you don't have to point it to the machine-specific libraries. You can simply create one library (ld -dylib) re-exporting exactly what you would have done when linking it in R (i.e. use -reexport-l instead of -l).

For ATLAS it's much easier since you can compile it statically with PIC and create one library -- e.g. see
http://r.research.att.com/libatlas-MacPro.dylib
(Note that one is 10.6 only but includes both 32-bit and 64-bit ATLAS 3.9.25)

Cheers,
Simon

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Michael Braun
    

    

      
      #
    

  


  

      
Simon:

Thanks for sending this over.   I'd much rather stick with the CRAN binary if I can, but exploiting the capabilities of my machine as much as possible is important to me.  There are 3 reasons I'm not using the precompiled Mac binary:

1.  Linking to MKL Blas
2.  Wanted to have a version of R where the linker lets me profile my precompiled code (you and I corresponded in a thread about this last year).
3. Compiling R for the newest Nehalem, and the vectorizing operations, using the -msse4.2 flag.

I've still never been able to get #2 to work, and it's really not a huge priority at the moment.   #3 would be nice if it does let R milk all the power out of the new processor (I've never done a speed test on it).  But if I understand correctly, would I now be able to use the CRAN R binary and still choose my own BLAS?  Whether I choose MKL or ATLAS, I'm sorry that I don't really understand what you are talking about when you say :

"MKL is picking them at run time so you don't have to point it to the machine-specific libraries. You can simply create one library (ld -dylib) re-exporting exactly what you would have done when linking it in R (i.e. use -reexport-l instead of -l)." 

I didn't mean pointing to the machine-specific libraries.  In the MKL Users Manual (July 2010 version, Chapter 5), there are different options depending on if you want to link static or dynamic, sequential or threaded, ILP or LP, FORTRAN interface or not, etc.  That;s why my Makevars file looks like:  BLAS_LIBS=-lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread  (and incidentally, adding -lpthread is mentioned in the Intel manual, even for the sequential version.)

Since I'm learning this all as I go, could you possibly provide a cookbook recipe of exactly what I would need to do.  I'm just not sure what I should be re-exporting.  Is this a new script file?  Or something I need to compile?

Another issue I've considered when choosing the MKL BLAS:  sometimes I'm calling BLAS routines from sequential code, in which case the threaded BLAS makes sense.  But sometimes I call from threaded code, either using the plyr, doMC, foreach, and /or multicore R packages, or using OpenMP.  In those cases, I've been setting MKL_NUM_THREADS (separately from OMP_NUM_THREADS) as needed.  I have not found a comparable way to do this in the ATLAS manual.

Thanks,

Michael

        
On Oct 12, 2010, at 11:44 AM, Simon Urbanek wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        
-------------------------------------------
Michael Braun
Homer A. Burnell (1928) Career Development Professor, 
	and Assistant Professor of Management Science (Marketing Group)
MIT Sloan School of Management
100 Main St.., E62-535
Cambridge, MA 02139
braunm at mit.edu
617-253-3436

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      Michael Braun
    

    

      
      #
    

  


  

      
OK, here's what I did, and it seems to work:

1.  I started with a clean R installation, using only the CRAN binary.

2.  I ran the following script:

#!/bin/bash

export BL=/opt/intel/Compiler/11.1/089/Frameworks/mkl/Libraries/em64t
export GF=~/Install_Files/MKL_gfortran_interface/lib/em64t
export IOMP=/opt/intel/Compiler/11.1/089/lib

ld -dylib -arch x86_64 -L$BL -L$GF -reexport-lmkl_intel_lp64 -reexport-lmkl_sequential -reexport-lmkl_core -reexport-lpthread -o libmklblas_seq.dylib
ld -dylib -arch x86_64 -L$BL -L$GF -L$IOMP  -reexport-lmkl_intel_lp64 -reexport-lmkl_intel_thread  -reexport-lmkl_core -reexport-liomp5 -reexport-lpthread -o libmklblas_par.dylib

This gave me two Intel MKL libraries.  I then pointed the libRblas.dylib symbolic link to one or the other, depending on if I want a sequential or threaded.
The performance of the threaded MKL is comparable to ATLAS (sometimes better, sometimes worse), but is much, much better than vecLib.  

So this gives me the BLAS I want, and keeps my life very simple by letting me use the CRAN binary.

But as researchers, we know that every answer spawns new questions.

1.  Can I do something similar with LAPACK (say, if I want a threaded Cholesky decomposition or solving a system of equations)?

2.  Although it is true that one can adjust the number of threads that the MKL BLAS uses by setting the MKL_NUM_THREADS environment variable, it appears that this variable must be set before the BLAS is loaded.  Since BLAS is loaded at the same time as R, it would appear that this kind of adaptation "on the fly" is not possible.  Is there a way to "reset" the BLAS at a suitable point in an R script, so I could unload BLAS, change the variable, and then reload it? 

The *specific* case I'm talking about is what would happen if I were to call multithreaded BLAS from a function that is running via multicore, foreach and/or plyr?  I'm thinking I want threading turned on when I'm not in a foreach loop, but off when I am.

3.  Do I have to worry about the MKL Fortran interfaces to BLAS or LAPACK at all?  (I'm talking about the libmkl_blas95_lp64.a and libmkl_lapack95_lp64.a libraries).  I needed to link to them to compile R from source, but if I omit them from today's new creation, will that mess me up down the road?

4.  As I mentioned before, I'm learning a lot this as I go, but from what I've read, the newer Intel processors support a number of vectorization instructions that are not supported by earlier chips (this SSE4.2 stuff).  I have tried compiling other packages (e.g., GSL) with and without SSE4.2, and find that adding the appropriate flag in the Intel compiler gives me 10-20 percent improvement in some applications (can't remember which--did it a while ago).  So that's why I add those flags when I compile R from source.  But I recognize that the CRAN binary needs to have some backwards compatibility.  The Intel compilers offer such an option (e.g., -axsse4.2 will compile to use the best possible set of instructions up to sse4.2).  Is the CRAN binary compiled in the same way? Put another way, is it compiled for the "lowest" common instruction set, or will it use the most advanced instruction set if it's available?

As before, I am happy to split these questions into separate discussion threads.  But now the questions are more of the "this is interesting--I'd like to learn more," variety, instead of the "I have a problem, please help me" type.  And of course, thanks so much for pointing me in the right direction, and for all the work you do in developing and supporting R.

        
On Oct 12, 2010, at 11:44 AM, Simon Urbanek wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        
-------------------------------------------
Michael Braun
Homer A. Burnell (1928) Career Development Professor, 
	and Assistant Professor of Management Science (Marketing Group)
MIT Sloan School of Management
100 Main St.., E62-535
Cambridge, MA 02139
braunm at mit.edu
617-253-3436