Installing Rmpi on 64-bit Linux Athlon
Many thanks for help on this. Using Hin-Tak's (offline) suggestion, I was able to get Rmpi to compile on R 2.4.0 by manually linking the offending code using a different shared library (which didn't seem to be picked up with any of the command-line arguments), and then re-installing the usual way. setenv LDFLAGS "-L$prefix/lib -L$prefix/lib64" R CMD INSTALL ~/code/Rmpi --configure-args="--with-mpi=$prefix --prefix=$prefix CC=mpicc" -l $R_LIBS cd ~/code/Rmpi/src gcc -shared -L$prefix/lib -o Rmpi.so RegQuery.o Rmpi.o conversion.o internal.o -lmpi -llam -lutil -lpthread R CMD INSTALL ~/code/Rmpi --configure-args="--with-mpi=$prefix CC=mpicc" -l $R_LIBS Once the package was installed, I quickly found out that Rmpi simply does not work under R 2.4.0 (for me at least), partly because of a change in the serialize function in R 2.4.0 (affecting bin.nchar()), and partly for reasons I haven't been able to figure out. (Looks like a failure in the mpi_allgather function in Rmpi.c, but am not sure). So for now, I'm retaining R 2.3.1 for parallel processing work. If anyone succeeds in compiling Rmpi on 2.4.0, please pass along any necessary changes to the package maintainer. Best, Robert -----Original Message----- From: Hin-Tak Leung [mailto:hin-tak.leung at cimr.cam.ac.uk] Sent: Tuesday, October 10, 2006 6:10 AM To: c.farrow at compserv.gla.ac.uk Cc: McGehee, Robert; r-devel at stat.math.ethz.ch Subject: Re: [Rd] Installing Rmpi on 64-bit Linux Athlon No, both of you are not doing it correctly - you are confused about linker problem with runtime issue. LD_LIBRARY_PATH is a runtime flag. You want to set the linker flag to influence the linker behavior: setenv LDFLAGS="-L$prefix/lib -L$prefix/lib64" or edit the makefile or Makevars do the equivalent with PKG_LDFLAGS in the Rmpi package. BTW, the syntax for using LD_LIBRARY_PATH is also wrong - to use it the way it is intended, you need: setenv LD_LIBRARY_PATH "$prefix/lib:$prefix/lib64"
c.farrow at compserv.gla.ac.uk wrote:
Hi, I think you need to reverse the order of the first two lines as currently you are using $prefix before it is defined. setenv prefix "/home/a347549/local64" setenv LD_LIBRARY_PATH "-L$prefix/lib -L$prefix/lib64" colin. On 9 Oct, McGehee, Robert wrote:
Hello, We have recently added a 64-bit 2 x Dual-Core Athlon server running
Red
Hat Enterprise Linux AS release 3 to allow for processing of large
data
sets (>4GB) in R. To integrate this server into our Linux cluster, I have been trying (unsuccessfully) to use Rmpi/lam-mpi to parallel process some of our scripts. While I have successfully compiled R
2.4.0
and lam-mpi 7.1.2 on this server using a 64-bit architecture, I have
not
been able to install Rmpi to interface with lam-mpi, despite success
on
nearly identical servers in a 32-bit architecture. When trying on the 64-bit server, I receive the error "cannot find -lmpi" during compilation (see below). If anyone has had success in this regards and notice an installation problem that I may have overlooked, I would greatly appreciate any advice. Note that because I do not have root access, all of the
lam-mpi
and R files were installed locally. Thanks in advance, Robert ### Below is configuration scripts for R, Lam-MPI and Rmpi: setenv LD_LIBRARY_PATH "-L$prefix/lib -L$prefix/lib64" setenv prefix "/home/a347549/local64" ### R 2.4.0 Configure ./configure --prefix=$prefix --with-x --with-tcl-config="$prefix/lib/tclConfig.sh" --with-tk-config="$prefix/lib/tkConfig.sh" --with-blas="-L$builddir/ATLAS/lib/Linux_HAMMER64SSE2_4 -lptf77blas -lpthread -latlas" ### Lam-MPI 7.1.2 Configure: ./configure --prefix=$prefix --enable-shared --disable-static --without-romio --with-rsh="ssh -xq" ### Rmpi INSTALL R CMD INSTALL Rmpi --configure-args="--with-mpi=$prefix
--prefix=$prefix
CC=mpicc" -l $R_LIBS * Installing *source* package 'Rmpi' ... Try to find mpi.h ... Found in /gcm/home/a347549/local64/include Try to find libmpi or libmpich ... checking for main in -lmpi... yes Try to find liblam ... checking for main in -llam... yes checking for openpty in -lutil... yes checking for main in -lpthread... yes configure: creating ./config.status config.status: creating src/Makevars ** libs gcc -I/gcm/home/a347549/local64/lib64/R/include -I/gcm/home/a347549/local64/lib64/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -I/gcm/home/a347549/local64/include -DMPI2 -I/usr/local/include -fpic -g -O2 -std=gnu99 -c RegQuery.c -o RegQuery.o gcc -I/gcm/home/a347549/local64/lib64/R/include -I/gcm/home/a347549/local64/lib64/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -I/gcm/home/a347549/local64/include -DMPI2 -I/usr/local/include -fpic -g -O2 -std=gnu99 -c Rmpi.c -o Rmpi.o gcc -I/gcm/home/a347549/local64/lib64/R/include -I/gcm/home/a347549/local64/lib64/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -I/gcm/home/a347549/local64/include -DMPI2 -I/usr/local/include -fpic -g -O2 -std=gnu99 -c conversion.c -o conversion.o gcc -I/gcm/home/a347549/local64/lib64/R/include -I/gcm/home/a347549/local64/lib64/R/include -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -I/gcm/home/a347549/local64/include -DMPI2 -I/usr/local/include -fpic -g -O2 -std=gnu99 -c internal.c -o internal.o gcc -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o Rmpi.o
conversion.o
internal.o -lmpi -llam -lutil -lpthread /usr/bin/ld: cannot find -lmpi collect2: ld returned 1 exit status make: *** [Rmpi.so] Error 1 chmod: cannot access `/home/a347549/code/rlib64/Rmpi/libs/*': No such file or directory ERROR: compilation failed for package 'Rmpi' ** Removing '/home/a347549/code/rlib64/Rmpi' [lwtc5.fmr.com/A347549] /home/a347549/code ############################################################ In addition, here is the output of Rmpi's config.log ############################################################
cat config.log
This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by configure, which was generated by GNU Autoconf 2.59. Invocation command line was $ ./configure --with-mpi=/gcm/home/a347549/local64 --prefix=/gcm/home/a347549/local64 CC=mpicc ## --------- ## ## Platform. ## ## --------- ## hostname = lwtc5.fmr.com uname -m = x86_64 uname -r = 2.6.9-34.0.1.ELsmp uname -s = Linux uname -v = #1 SMP Wed May 17 16:59:36 EDT 2006 /usr/bin/uname -p = unknown /bin/uname -X = unknown /bin/arch = x86_64 /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown PATH: /home/a347549/local64/java/bin PATH: /home/a347549/local64/tibrv/bin PATH: /home/a347549/local64/ant/bin PATH: /home/a347549/local64/bin PATH: /usr/bin/X11 PATH: /home/a347549/bin PATH: /home/a347549/local64/java/bin PATH: /home/a347549/local64/tibrv/bin PATH: /home/a347549/local64/ant/bin PATH: /home/a347549/local64/bin PATH: /usr/bin/X11 PATH: /home/a347549/bin PATH: /usr/kerberos/bin PATH: /usr/local/bin PATH: /bin PATH: /usr/bin PATH: /usr/X11R6/bin PATH: . PATH: . ## ----------- ## ## Core tests. ## ## ----------- ## configure:2944: checking for main in -lmpi configure:2968: mpicc -o conftest conftest.c -lmpi >&5 configure:2974: $? = 0 configure:2978: test -z || test ! -s conftest.err configure:2981: $? = 0 configure:2984: test -s conftest configure:2987: $? = 0 configure:3000: result: yes configure:3018: checking for main in -llam configure:3042: mpicc -o conftest conftest.c -llam >&5 configure:3048: $? = 0 configure:3052: test -z || test ! -s conftest.err configure:3055: $? = 0 configure:3058: test -s conftest configure:3061: $? = 0 configure:3074: result: yes configure:3086: checking for openpty in -lutil configure:3116: mpicc -o conftest conftest.c -lutil >&5 configure:3122: $? = 0 configure:3126: test -z || test ! -s conftest.err configure:3129: $? = 0 configure:3132: test -s conftest configure:3135: $? = 0 configure:3148: result: yes configure:3154: checking for main in -lpthread configure:3178: mpicc -o conftest conftest.c -lpthread >&5 configure:3184: $? = 0 configure:3188: test -z || test ! -s conftest.err configure:3191: $? = 0 configure:3194: test -s conftest configure:3197: $? = 0 configure:3210: result: yes configure:3352: creating ./config.status ## ---------------------- ## ## Running config.status. ## ## ---------------------- ## This file was extended by config.status, which was generated by GNU Autoconf 2.59. Invocation command line was CONFIG_FILES = CONFIG_HEADERS = CONFIG_LINKS = CONFIG_COMMANDS = $ ./config.status on lwtc5.fmr.com config.status:635: creating src/Makevars ## ---------------- ## ## Cache variables. ## ## ---------------- ## ac_cv_env_CC_set=set ac_cv_env_CC_value=mpicc ac_cv_env_CFLAGS_set= ac_cv_env_CFLAGS_value= ac_cv_env_CPPFLAGS_set= ac_cv_env_CPPFLAGS_value= ac_cv_env_CPP_set= ac_cv_env_CPP_value= ac_cv_env_LDFLAGS_set= ac_cv_env_LDFLAGS_value= ac_cv_env_build_alias_set= ac_cv_env_build_alias_value= ac_cv_env_host_alias_set= ac_cv_env_host_alias_value= ac_cv_env_target_alias_set= ac_cv_env_target_alias_value= ac_cv_lib_lam_main=yes ac_cv_lib_mpi_main=yes ac_cv_lib_pthread_main=yes ac_cv_lib_util_openpty=yes ## ----------------- ## ## Output variables. ## ## ----------------- ## CC='mpicc' CFLAGS='' CPP='' CPPFLAGS='' DEFS='-DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\"
-DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" '
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP='grep -E'
EXEEXT=''
LDFLAGS=''
LIBOBJS=''
LIBS=''
LTLIBOBJS=''
OBJEXT=''
PACKAGE_BUGREPORT=''
PACKAGE_NAME=''
PACKAGE_STRING=''
PACKAGE_TARNAME=''
PACKAGE_VERSION=''
PATH_SEPARATOR=':'
PKG_CPPFLAGS='-I/gcm/home/a347549/local64/include -DMPI2'
PKG_LIBS='-lmpi -llam -lutil -lpthread'
SHELL='/bin/sh'
ac_ct_CC=''
bindir='${exec_prefix}/bin'
build_alias=''
datadir='${prefix}/share'
exec_prefix='${prefix}'
host_alias=''
includedir='${prefix}/include'
infodir='${prefix}/info'
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localstatedir='${prefix}/var'
mandir='${prefix}/man'
oldincludedir='/usr/include'
prefix='/gcm/home/a347549/local64'
program_transform_name='s,x,x,'
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
sysconfdir='${prefix}/etc'
target_alias=''
## ----------- ##
## confdefs.h. ##
## ----------- ##
#define PACKAGE_BUGREPORT ""
#define PACKAGE_NAME ""
#define PACKAGE_STRING ""
#define PACKAGE_TARNAME ""
#define PACKAGE_VERSION ""
configure: exit 0
Robert McGehee
Quantitative Analyst
Geode Capital Management, LLC
53 State Street, 5th Floor | Boston, MA | 02109
Tel: 617/392-8396 Fax:617/476-6389
mailto:robert.mcgehee at geodecapital.com
This e-mail, and any attachments hereto, are intended for
us...{{dropped}}
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel