LinkingTo and C++

Hello,

I've been trying to make LinkingTo work when the package linked to has
c++ code.

I've put dumb packages to illustrate this emails here ;
http://addictedtor.free.fr/misc/linkingto

Package A defines this C++ class:

class A {
public:
A() ;
~A() ;
SEXP hello() ;
} ;

Package B has this function :

SEXP say_hello(){
A a ;
return a.hello() ;
}

headers of package A are copied into inst/include so that package B can
have.

LinkingTo: A

in its DESCRIPTION file.

Also, package B has the R function ;

g <- function(){
.Call("say_hello", PACKAGE = "B")
}

With this I can compile A and B, but then I get :

$ Rscript -e "B::g()"
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared library '/usr/local/lib/R/library/B/libs/B.so':
/usr/local/lib/R/library/B/libs/B.so: undefined symbol: _ZN1AD1Ev
Calls: :: ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>

If I then add a Makevars in B with this :


# find the root directory where A is installed
ADIR=$(shell $(R_HOME)/bin/Rscript -e "cat(system.file(package='A'))" )

PKG_LIBS= $(ADIR)/libs/A$(DYLIB_EXT)


Then it works:

$ Rscript -e "B::g()"
[1] "hello"

So it appears that adding the -I flag, which is what LinkingTo does is
not enough when the package "linking from" (B) actually has to link to
the "linked to" package (A).

I've been looking at
http://cran.r-project.org/doc/manuals/R-exts.html#Registering-native-routines
but it seems only applicable to c(++) functions and not classes ...

What am I missing ? Should/can linkingto be extended in a way that
accomodates c++

Romain

library (B)

.Call("say_hello", PACKAGE = "B")

Hello,

I've been trying to make LinkingTo work when the package linked to has c++ code.

I've put dumb packages to illustrate this emails here ; http://addictedtor.free.fr/misc/linkingto

Package A defines this C++ class:

class A {
public:
	A() ;
	~A() ;
	SEXP hello() ;
} ;

Package B has this function :

SEXP say_hello(){
	A a ;
	return a.hello() ;
}

headers of package A are copied into inst/include so that package B can have.

LinkingTo: A

in its DESCRIPTION file.

Also, package B has the R function ;

g <- function(){
	.Call("say_hello", PACKAGE = "B")
}

With this I can compile A and B, but then I get :

$ Rscript -e "B::g()"
Error in dyn.load(file, DLLpath = DLLpath, ...) :
 unable to load shared library '/usr/local/lib/R/library/B/libs/B.so':
 /usr/local/lib/R/library/B/libs/B.so: undefined symbol: _ZN1AD1Ev
Calls: :: ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>

If I then add a Makevars in B with this :


# find the root directory where A is installed
ADIR=$(shell $(R_HOME)/bin/Rscript -e "cat(system.file(package='A'))" )

PKG_LIBS= $(ADIR)/libs/A$(DYLIB_EXT)


Then it works:

$ Rscript -e "B::g()"
[1] "hello"

So it appears that adding the -I flag, which is what LinkingTo does is not enough when the package "linking from" (B) actually has to link to the "linked to" package (A).

I've been looking at http://cran.r-project.org/doc/manuals/R-exts.html#Registering-native-routines but it seems only applicable to c(++) functions and not classes ...

What am I missing ? Should/can linkingto be extended in a way that accomodates c++

Romain


-- 
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://tr.im/NrTG : Rcpp 0.7.5
|- http://tr.im/MPYc : RProtoBuf: protocol buffers for R
`- http://tr.im/KfKn : Rcpp 0.7.2

Romain,

I think your'e confusing two entirely different concepts here:

1) LinkingTo: allows a package to provide C-level functions to other packages (see R-ext 5.4). Let's say package A provides a function foo by calling R_RegisterCCallable for that function. If a package B wants to use that function, it uses LinkingTo: and calls R_GetCCallable to obtain the function pointer. It does not actually link to package A because that is in general not possible - it simply obtains the pointers through R. In addition, LinkingTo: makes sure that you have access to the header files of package A which help you to cast the functions and define any data structures you may need. Since C++ is a superset of C you can use this facility with C++ as long as you don't depend on anything outside of the header files.

2) linking directly to another package's shared object is in general not possible, because packages are not guaranteed to be dynamic libraries. They are usually shared objects which may or may not be compatible with a dynamic library on a given platform. Therefore the R-ext describes other way in which you may provide some library independently of the package shared object to other packages (see R-ext 5.8). The issue is that you have to create a separate library (PKG/libs[/arch]/PKG.so won't work in general!) and provide this to other packages. As 5.8 says, this is in general not trivial because it is very platform dependent and the most portable way is to offer a static library.

To come back to your example, LinkingTo: A and B will work if you remove Makevars from B (you don't want to link)and put your hello method into the A.h header:

library (B)

Loading required package: A

.Call("say_hello", PACKAGE = "B")

[1] "hello"

However, your'e not really using the LinkingTo: facilities for the functions so it's essentially just helping you to find the header file.

Cheers,
Simon



On Feb 11, 2010, at 4:08 AM, Romain Francois wrote:

Hello,

I've been trying to make LinkingTo work when the package linked to has c++ code.

I've put dumb packages to illustrate this emails here ; http://addictedtor.free.fr/misc/linkingto

Package A defines this C++ class:

class A {
public:
	A() ;
	~A() ;
	SEXP hello() ;
} ;

Package B has this function :

SEXP say_hello(){
	A a ;
	return a.hello() ;
}

headers of package A are copied into inst/include so that package B can have.

LinkingTo: A

in its DESCRIPTION file.

Also, package B has the R function ;

g<- function(){
	.Call("say_hello", PACKAGE = "B")
}

With this I can compile A and B, but then I get :

$ Rscript -e "B::g()"
Error in dyn.load(file, DLLpath = DLLpath, ...) :
  unable to load shared library '/usr/local/lib/R/library/B/libs/B.so':
  /usr/local/lib/R/library/B/libs/B.so: undefined symbol: _ZN1AD1Ev
Calls: :: ... tryCatch ->  tryCatchList ->  tryCatchOne ->  <Anonymous>

If I then add a Makevars in B with this :


# find the root directory where A is installed
ADIR=$(shell $(R_HOME)/bin/Rscript -e "cat(system.file(package='A'))" )

PKG_LIBS= $(ADIR)/libs/A$(DYLIB_EXT)


Then it works:

$ Rscript -e "B::g()"
[1] "hello"

So it appears that adding the -I flag, which is what LinkingTo does is not enough when the package "linking from" (B) actually has to link to the "linked to" package (A).

I've been looking at http://cran.r-project.org/doc/manuals/R-exts.html#Registering-native-routines but it seems only applicable to c(++) functions and not classes ...

What am I missing ? Should/can linkingto be extended in a way that accomodates c++

Romain

On Feb 11, 2010, at 12:24 PM, Romain Francois wrote:

Thanks.

On 02/11/2010 05:55 PM, Simon Urbanek wrote:

Romain,

I think your'e confusing two entirely different concepts here:

Yes. The name "LinkingTo" probably helped my confusion.

Admittedly, it's probably not the best name ;).

1) LinkingTo: allows a package to provide C-level functions to other packages (see R-ext 5.4). Let's say package A provides a function foo by calling R_RegisterCCallable for that function. If a package B wants to use that function, it uses LinkingTo: and calls R_GetCCallable to obtain the function pointer. It does not actually link to package A because that is in general not possible - it simply obtains the pointers through R. In addition, LinkingTo: makes sure that you have access to the header files of package A which help you to cast the functions and define any data structures you may need. Since C++ is a superset of C you can use this facility with C++ as long as you don't depend on anything outside of the header files.

2) linking directly to another package's shared object is in general not possible, because packages are not guaranteed to be dynamic libraries. They are usually shared objects which may or may not be compatible with a dynamic library on a given platform. Therefore the R-ext describes other way in which you may provide some library independently of the package shared object to other packages (see R-ext 5.8). The issue is that you have to create a separate library (PKG/libs[/arch]/PKG.so won't work in general!) and provide this to other packages. As 5.8 says, this is in general not trivial because it is very platform dependent and the most portable way is to offer a static library.

To come back to your example, LinkingTo: A and B will work if you remove Makevars from B (you don't want to link)and put your hello method into the A.h header:

Sure. but in real life I can't realistically put everything in the header files.

It was just an example based on your example ;) - which was not very realistic, either. It practice it is reasonable, because it is sufficient to declare in the headers whatever you're providing so the only homework is to cast function pointers you have obtained via R_GetCCallable to the declarations from the header file.

I suspect what you meant is not as much related to LinkingTo: (since the mess C++ creates at the binary level is rather hard to pass through dl pointers - but if someone has a working solution it may be worth to create a package), but rather to provide a library. That is not covered by R at this point so you're in realm of R-ext 5.8. Given how non-trivial task this is (to get it right) it may be worthwhile thinking about a portable solution and add it to R, but I don't think anyone has done that yet (mainly due to the low benefit/cost ratio I suspect). For all cases so far it was sufficient to create C or R level API for other package to use.

Cheers,
Simon