Skip to content

[R-sig-dyn-mod] Problem with loading compiled code into R

8 messages · Hagens, M. (Mathilde), John Harrold, Tim Keitt +1 more

Original
Hagens, M. (Mathilde)
# 
Dear all,

I?m using compiled code (written in C) in combination with the deSolve package to run a one-dimensional reactive transport model in R. On a server I use running on Linux this works flawlessly, but on a MacBook Pro (macOS 10.12.4) I have problems loading the compiled code into R (version 3.3.3).

The output I get when compiling the code in R using
system("R CMD SHLIB Model.c?)
is:
clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c Model.c -o Model.o
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o Model.so Model.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation

However, when I try to load the resulting shared object via
dyn.load(paste("Model", .Platform$dynlib.ext, sep = ""))
I get:
Error in dyn.load(paste("Model", .Platform$dynlib.ext, sep = "")) :
  unable to load shared object '/Users/mathilde/Model.so':
  dlopen(/Users/mathilde/Model.so, 6): Symbol not found: _dH_dt
  Referenced from: /Users/mathilde/Model.so
  Expected in: flat namespace
 in /Users/mathilde/Model.so

The shared object created after compiling seems to look fine. Also, I tried compiling directly from the Terminal, but that doesn?t make a difference. Strangely, when I ran this model on a Mac in the past (~1 year ago) it also worked without problems, leaving me wondering whether it may be due to an R / macOS update. However, switching to an older R version (2.15.2) didn?t solve the issue either. Has anyone experienced similar behaviour and/or knows of a possible solution?

Best wishes,
Mathilde
Thomas Petzoldt
# 
Hi Mathilde,

a short question: which version of deSolve did you use? The stable 
version 1.14 from CRAN (i.e. the usual case) or a development snapshot 
from R-Forge (currently 1.20)?

Can you test your model on a Windows or Linux system?

Another idea may be a change of the compiler on the Mac. The CRAN tests 
of deSolve (1.14) are ok for the Mac, but I have personally no Mac at 
hand for testing. Is anybody else able to check this, or provide 
information about changes of the C compilers on the Mac?

You may also consider to ask the R-Sig-Mac mailing list:
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Thomas

Am 07.04.2017 um 12:20 schrieb Hagens, M. (Mathilde):

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      John Harrold
    

    

      
      #
    

  


  

      
Howdy Mathilde,

I have a Mac and I run compiled code in R using deSolve all the time.
If you can send me your C file and an R script to compile, load, and
run it, I can give it a try on my computer.

John

On Fri, Apr 7, 2017 at 10:36 AM, Thomas Petzoldt

        
<thomas.petzoldt at tu-dresden.de> wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Tim Keitt
    

    

      
      #
    

  


  

      
On Fri, Apr 7, 2017 at 5:20 AM, Hagens, M. (Mathilde) <M.Hagens at uu.nl>
wrote:

            

      
      
      
    

        
If you google that error message, you will find the answer. (clang v. gcc)

THK

http://www.keittlab.org/

  
  


  


      

    

    

      
      

        


  

        

      Tim Keitt
    

    

      
      #
    

  


  

      
On Fri, Apr 7, 2017 at 10:08 PM, Tim Keitt <tkeitt at utexas.edu> wrote:

        

            

      
      
      
    

        
Let me clarify as I was alerted that that sounded harsh. (Not my intention.)

This error is often reported when linking code C++ compiled with gcc to
code compiled with clang. Those don't work well together. If your R was
compiled with gcc (clang), then you cannot use clang (gcc) to compile code
to link with R (even the g++-alike that comes with llvm).

The trick is to use "R CMD config" to find the compiler used to compile R.

In R (for the C++ compiler):

            

      
      
      
    

        
After that, "R CMD SHLIB --clean" should use the correct compiler and link
properly.

You can also try declaring your call point "extern C" to use the C linkage
ABI.

If you're using C rather than C++, then I'm not sure why it does not work
as the C linkage ABI is supposed to be standardized. You can try the above
with CC in place of CXX. Maybe that will fix it.

THK

http://www.keittlab.org/

  
  


  


      

    

    

      
      

        


  

        

      Hagens, M. (Mathilde)
    

    

      
      #
    

  


  

      
Tim, thank you for your clarification. (I did not consider your earlier message harsh.)
I actually had been looking at this, also before posting on the mailing list (maybe I should have mentioned that). My R is compiled using clang, the default for the current CRAN distribution for macOS (see https://cran.r-project.org/doc/manuals/r-release/R-admin.html#macOS). Since the compiler I used was also clang, this does not directly seem to be the problem. However, I am still playing around with some of the compiler flags to see if they may have caused the issue. For this, I find it very helpful to have a look at the Makeconf in R, under file.path(R.home("etc"), "Makeconf?). If anyone has some ideas on what I could change there, that would be highly appreciated as I am still learning about this.
FYI, your example does work with CC instead of CXX.

Thomas, I use deSolve 1.14 and R 3.3.3. My model runs flawlessly on a Linux server (with deSolve 1.14 and R 3.3.1). I don?t have easy access to a Windows computer at the moment. But right now I definitely believe it is a compiler issue.

Once I have a solution (hopefully) I?ll post it here.

Thanks for all your input!
Mathilde

Dr M. (Mathilde) Hagens | Postdoc | Department of Earth Sciences - Geochemistry, Faculty of Geosciences | Utrecht University | Princetonplein 9, 3584 CC Utrecht, the Netherlands | Room 1.13 | P.O. Box 80.021, 3508 TA Utrecht, the Netherlands | tel. +31 30 253 50 37 | M.Hagens at uu.nl<mailto:M.Hagens at uu.nl> | www.uu.nl/staff/MHagens<http://www.uu.nl/staff/MHagens> |

        
On 8 Apr 2017, at 19:23, Tim Keitt <tkeitt at utexas.edu<mailto:tkeitt at utexas.edu>> wrote:

        

        
On Fri, Apr 7, 2017 at 10:08 PM, Tim Keitt <tkeitt at utexas.edu<mailto:tkeitt at utexas.edu>> wrote:

        
If you google that error message, you will find the answer. (clang v. gcc)


Let me clarify as I was alerted that that sounded harsh. (Not my intention.)

This error is often reported when linking code C++ compiled with gcc to
code compiled with clang. Those don't work well together. If your R was
compiled with gcc (clang), then you cannot use clang (gcc) to compile code
to link with R (even the g++-alike that comes with llvm).

The trick is to use "R CMD config" to find the compiler used to compile R.

In R (for the C++ compiler):

Sys.setenv(CXX = system2("R", "CMD config CXX", stdout = TRUE))
Sys.which(Sys.getenv("CXX"))  # should return compiler path

After that, "R CMD SHLIB --clean" should use the correct compiler and link
properly.

You can also try declaring your call point "extern C" to use the C linkage
ABI.

If you're using C rather than C++, then I'm not sure why it does not work
as the C linkage ABI is supposed to be standardized. You can try the above
with CC in place of CXX. Maybe that will fix it.

THK

http://www.keittlab.org/


_______________________________________________
R-sig-dynamic-models mailing list
R-sig-dynamic-models at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-dynamic-models

  
  


  


      

    

    
3 days later

    

      
      

        


  

        

      Thomas Petzoldt
    

    

      
      #
    

  


  

      
Hi,

many thanks to Tim for the detailed answer and to Mathilde for further 
clarification.

Just for the record, in addition to the CRAN tests we tested deSolve 
1.14 with CLANG and a "user-compiled" model (see attachment):

- on Ubuntu 16.04 (R-devel) and

- macOS 10.12.4 (R 3.3.2)
    (on the laptop of our master student Felix)

The example compiled, loaded and worked flawlessly without any problems.

Thomas



-------------- next part --------------
library(deSolve)
library(scatterplot3d)

parameters <- c(a = -8/3, b = -10, c =  28)
state <- c(X = 1, Y = 1, Z = 1)
times <- seq(0, 100, by = 0.01)


system("R CMD SHLIB lorenzc.c")

dll <- paste0("lorenzc", .Platform$dynlib.ext)

dyn.load(dll)

out <- ode(state, times, func = "derivs", parms = parameters,
      dllname = "lorenzc", initfunc = "initmod")

dyn.unload(dll)


plot(out)

library(scatterplot3d)
scatterplot3d(out[,-1], type="l")

-------------- next part --------------
/* file lorenz.c */
#include <R.h>
static double parms[3];
#define a parms[0]
#define b parms[1]
#define c parms[2]

/* initializer  */
void initmod(void (* odeparms)(int *, double *))
{
    int N = 3;
    odeparms(&N, parms);
}

/* Derivatives */
void derivs (int *neq, double *t, double *y, double *ydot,
             double *yout, int *ip)
{
    ydot[0] = a * y[0] + y[1] * y[2];
    ydot[1] = b * (y[1] - y[2]);
    ydot[2] = - y[0] * y[1] + c * y[1] - y[2];
}

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      Hagens, M. (Mathilde)
    

    

      
      #
    

  


  

      
Hi all,

Thank you for sharing these files Thomas. I could run your example without problem, leading me to dig into the C code again. It turned out that I got it up and running on both Mac and Linux after declaring one of my routines in C as ?void? rather than ?inline void?. However, it still leaves me completely puzzled why the previous version of my code did work on the Linux server (I have again checked that this is the case). In any case, I learned a lot about compiler flags through this process. Thanks everyone for the input.

Mathilde