Skip to content

Problems with runtime linking of R-extension

3 messages · Keith O'Hara, Ralf Stubner

#
Hi all,

I am trying to port RcppArrayFire
(https://github.com/RInstitute/rcpparrayfire) to Mac OS, but I am having
some troubles with resolving linked libraries. I have tried to reduce
the issue as much as possible, i.e. no Rcpp and no package structure. It
seems to be MacOS specific since the same code works on Linux (as does
RcppArrayFire). I also asked this on stackoverflow
(https://stackoverflow.com/questions/48705490/runtime-linking-r-extension-on-macos),
but that did not draw a lot of attention. Maybe someone on this list can
shed some light on this:

When I install ArrayFire (http://arrayfire.com/) on MacOS using the
binary installer, the libraries are installed in /usr/local/lib with an
install name starting with @rpath:

$ otool -L /usr/local/lib/libaf.dylib /usr/local/lib/libaf.dylib:
	@rpath/libaf.3.dylib (compatibility version 3.0.0, current version 3.5.1)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
120.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1226.10.1)

 I can compile, link and run simple examples, e.g.

#include <arrayfire.h>
#include <stdio.h>

int main() {
  unsigned int count;
  at_get_backend_count(&count);
  printf("backends: %d\n", count);
  return 0;
}

gives

$ /usr/local/clang4/bin/clang -laf -o minimal minimal.c
$ otool -L minimal
minimal:
	@rpath/libaf.3.dylib (compatibility version 3.0.0, current version 3.5.1)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.0.0)
$ ./minimal
backends: 2


To do the same from R I use

#include <arrayfire.h>
#include <Rinternals.h>
SEXP count_backends() {
  unsigned int count;
  af_get_backend_count(&count);
  Rprintf("backends: %d\n", count);
  return R_NilValue;
}

And compile it with

$ PKG_LIBS=-laf R CMD SHLIB minimal.c /usr/local/clang4/bin/clang
-I/Library/Frameworks/R.framework/Resources/include -DNDEBUG
-I/usr/local/include   -fPIC  -Wall -g -O2  -c minimal.c -o minimal.o
/usr/local/clang4/bin/clang -dynamiclib -Wl,-headerpad_max_install_names
-undefined dynamic_lookup -single_module -multiply_defined suppress
-L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang4/lib
-o minimal.so minimal.o -laf -F/Library/Frameworks/R.framework/..
-framework R -Wl,-framework -Wl,CoreFoundation
$ otool -L minimal.so  minimal.so:
	minimal.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libaf.3.dylib (compatibility version 3.0.0, current version 3.5.1)
	/Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libR.dylib
(compatibility version 3.4.0, current version 3.4.3)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1451.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.0.0)

If I then try to load the resulting library, I get this error message:

R> dyn.load("minimal.so")
Error in dyn.load("minimal.so") :   unable to load shared object
'/Users/ralf/Documents/af-simple/minimalR/minimal.so':
  dlopen(/Users/ralf/Documents/af-simple/minimalR/minimal.so, 6):
Library not loaded: @rpath/libaf.3.dylib
  Referenced from: /Users/ralf/Documents/af-simple/minimalR/minimal.so
  Reason: image not found
Execution halted


Both the binary and the R-extension library refer to the external
library using the same install name, but for the R-extension this
reference cannot be resolved. Why is this the case?

Thanks
Ralf

PS: Used versions:
* Mac OS High Siearra
* R 3.4.3
* clang 4.0.0 from https://cran.r-project.org/bin/macosx/tools/
#
Try adding '/usr/local/lib' to DYLD_LIBRARY_PATH and DYLD_FALLBACK_LIBRARY_PATH. If that doesn't fix the problem, try modifying the install name of libaf to include /usr/local/lib. Something like:

sudo install_name_tool -id /usr/local/lib/libaf.dylib /usr/local/lib/libaf.dylib

or

sudo install_name_tool -id /usr/local/lib/libaf.3.dylib /usr/local/lib/libaf.3.dylib

This has fixed runtime path issues for me in the past.

Keith
#
Unfortunately setting these variables (or LD_LIBRARY_PATH) to /usr/local/lib does not help with the error message.
This helps to some extend, similar to adding ?-rpath /usr/local/lib? to PKG_LIBS. I am then able to load the library. However, ArrayFire does not find/load its backend libraries libafcpu.dynlib and libafopencl.dynlib, which are also located in /usr/local/lib and therefore reports ?backends: 0". This does not happen with the plain C program that correctly finds two available backends.

To make things even stranger, the above is true when I start R from the Terminal. When using R.app or RStudio, I can load minimal.so (even without rpath option or manipulating the install name)  and calling the count_backends function gives the correct result. 

What does R.app and RStudio to differently from R in the Terminal?

Thanks
Ralf