[Rcpp-devel] Re-using C++ code from a package with modules and defining linker flags using Rscript

Hi Jelmer,

| Dear all,
| 
| This is a question about recommended ways of linking C++ code in an R
| package to C++ code defined in other R packages (using Rcpp 0.9.4.1

Well, you seem to make the assumption that this would be supported and
documented. But it is not really. 

| and Rtools on Windows). I built package A, which contains C++ classes
| and uses modules to expose some of the methods in these classes to R.
| Then I built package B, which uses some of the C++ classes in A
| directly. I added package A to the LinkingTo: and Depends: fields in
| DESCRIPTION. In order for R to find the header-files (.h) of package
| A, when compiling package B, I put those header files in
| A/inst/include. R-Functions LdFlags() and CxxFlags() are defined in
| package A, so I can have something like
| 
| PKG_LIBS = $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e
| "Rcpp:::LdFlags()") \
|            $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e "A:::LdFlags()")
| 
| in B/src/Makevars.
| 
| At this point I ran into problems when installing package B (error:
| failed to load module modA from package A). It turns out the same
| problem occurs when using a Rcpp skeleton package with modules.
| 
| library('Rcpp')
| Rcpp.package.skeleton( 'testmodA', module=TRUE )
| 
| with one file (e.g. inline.R) added in testmodA/R containing
| ===
| LdFlags <- function(print=TRUE) {
|     if (print) cat("test\n") else return( "test" )
| }
| ===
| 
| Install the package
| R CMD INSTALL testmodA
| 
| Including
| PKG_LIBS = $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e
| "Rcpp:::LdFlags()") \
|            $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e
| "testmodA:::LdFlags()")
| 
| in package B's Makevars leads to errors during compilation, which I
| traced back to:
| > Rscript -e "testmodA:::LdFlags()"
| 
| Error: .onLoad failed in loadNamespace() for 'testmodA', details:
|   call: value[[3L]](cond)
|   error: failed to load module yada from package testmodA
| Execution halted
| 
| If I replace this by
| Rscript -e "library(testmodA); testmodA:::LdFlags()"
| Loading required package: methods
| Loading required package: Rcpp
| test
| 
| it works as expected, so this is what I use in the Makevars file of package B.
| 
| Without modules in testmodA
| > Rscript -e "testmodA:::LdFlags()"
| works fine.
| 
| Is this the recommended way of adding linker and cxx flags to packages
| depending on other R packages with modules? I tried to look at

We simply show no example of definiting object code in one package and having
other packages use it -- with the exception of the Rcpp package itself.  And
there it takes quite some effort to make this happen.

You could look at what lme4 and Matrix do; this is documented in the 'Writing
R Extensions' manual under '[Section] 5.4 Registering native routines'.

| packages using RcppArmadillo or RcppGSL for inspiration, but both of
| these packages don't implement modules. Any other packages I should
| look at?

RcppArmadillo is very different: it used templates only, no linking.  RcppGSL
uses GSL; for that we standard configure and system GSL libraries.  None of
these package take anything from another R package, besides Rcpp.

So as a simpler step I would just include package A's code in package B, or
make it one package.

Sorry,  Dirk

Gauss once played himself in a zero-sum game and won $50.
                      -- #11 at http://www.gaussfacts.com

Hi Jelmer,

On 15 May 2011 at 21:42, Jelmer Ypma wrote:
| Dear all,
|
| This is a question about recommended ways of linking C++ code in an R
| package to C++ code defined in other R packages (using Rcpp 0.9.4.1

Well, you seem to make the assumption that this would be supported and
documented. But it is not really.

| and Rtools on Windows). I built package A, which contains C++ classes
| and uses modules to expose some of the methods in these classes to R.
| Then I built package B, which uses some of the C++ classes in A
| directly. I added package A to the LinkingTo: and Depends: fields in
| DESCRIPTION. In order for R to find the header-files (.h) of package
| A, when compiling package B, I put those header files in
| A/inst/include. R-Functions LdFlags() and CxxFlags() are defined in
| package A, so I can have something like
|
| PKG_LIBS = $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e
| "Rcpp:::LdFlags()") \
|            $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e

"A:::LdFlags()")

|
| in B/src/Makevars.
|
| At this point I ran into problems when installing package B (error:
| failed to load module modA from package A). It turns out the same
| problem occurs when using a Rcpp skeleton package with modules.
|
| library('Rcpp')
| Rcpp.package.skeleton( 'testmodA', module=TRUE )
|
| with one file (e.g. inline.R) added in testmodA/R containing
| ===
| LdFlags <- function(print=TRUE) {
|     if (print) cat("test\n") else return( "test" )
| }
| ===
|
| Install the package
| R CMD INSTALL testmodA
|
| Including
| PKG_LIBS = $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e
| "Rcpp:::LdFlags()") \
|            $(shell "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" -e
| "testmodA:::LdFlags()")
|
| in package B's Makevars leads to errors during compilation, which I
| traced back to:
| > Rscript -e "testmodA:::LdFlags()"
|
| Error: .onLoad failed in loadNamespace() for 'testmodA', details:
|   call: value[[3L]](cond)
|   error: failed to load module yada from package testmodA
| Execution halted
|
| If I replace this by
| Rscript -e "library(testmodA); testmodA:::LdFlags()"
| Loading required package: methods
| Loading required package: Rcpp
| test
|
| it works as expected, so this is what I use in the Makevars file of

package B.

|
| Without modules in testmodA
| > Rscript -e "testmodA:::LdFlags()"
| works fine.
|
| Is this the recommended way of adding linker and cxx flags to packages
| depending on other R packages with modules? I tried to look at

We simply show no example of definiting object code in one package and

having

other packages use it -- with the exception of the Rcpp package itself.

And

there it takes quite some effort to make this happen.

You could look at what lme4 and Matrix do; this is documented in the

'Writing

R Extensions' manual under '[Section] 5.4 Registering native routines'.

Unfortunately that approach doesn't work with C++ classes and methods, which
is one of the reasons that Rcpp linkage has to be done the way that it is.
The exporting of symbols a as done by Matrix is restricted to C function
entry points.

| packages using RcppArmadillo or RcppGSL for inspiration, but both of
| these packages don't implement modules. Any other packages I should
| look at?

RcppArmadillo is very different: it used templates only, no linking.

RcppGSL

uses GSL; for that we standard configure and system GSL libraries.  None

of

these package take anything from another R package, besides Rcpp.

So as a simpler step I would just include package A's code in package B,

or

make it one package.

Sorry,  Dirk

--
Gauss once played himself in a zero-sum game and won $50.
                     -- #11 at http://www.gaussfacts.com

_______________________________________________
Rcpp-devel mailing list
Rcpp-devel at lists.r-forge.r-project.org
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20110515/3ca06a1c/attachment-0001.htm>

| > | Is this the recommended way of adding linker and cxx flags to packages
| > | depending on other R packages with modules? I tried to look at
| >
| > We simply show no example of definiting object code in one package and having
| > other packages use it -- with the exception of the Rcpp package itself. ?And
| > there it takes quite some effort to make this happen.
| >
| > You could look at what lme4 and Matrix do; this is documented in the 'Writing
| > R Extensions' manual under '[Section] 5.4 Registering native routines'.
| 
| Unfortunately that approach doesn't work with C++ classes and methods, which is
| one of the reasons that Rcpp linkage has to be done the way that it is.? The
| exporting of symbols a as done by Matrix is restricted to C function entry
| points.

Ah, yes, that aspect too. Meant to mention, thanks for catching it.

In short: no beans.  You may have to build one larger package.

Dirk

Gauss once played himself in a zero-sum game and won $50.
                      -- #11 at http://www.gaussfacts.com

Thanks for both your replies. I got it working the way I wanted. If
someone's interested, a simple example with two packages can be
downloaded here: http://www.ucl.ac.uk/~uctpjyy/downloads/RcppEx.zip

QuadFunc defines a C++ class that is used in C++ code in QuadFuncVec.

Jelmer

On 15 May 2011 at 17:22, Douglas Bates wrote:
| On May 15, 2011 5:17 PM, "Dirk Eddelbuettel" <edd at debian.org> wrote:
| > On 15 May 2011 at 21:42, Jelmer Ypma wrote:
| > | Is this the recommended way of adding linker and cxx flags to packages
| > | depending on other R packages with modules? I tried to look at
| >
| > We simply show no example of definiting object code in one package and having
| > other packages use it -- with the exception of the Rcpp package itself. ?And
| > there it takes quite some effort to make this happen.
| >
| > You could look at what lme4 and Matrix do; this is documented in the 'Writing
| > R Extensions' manual under '[Section] 5.4 Registering native routines'.
|
| Unfortunately that approach doesn't work with C++ classes and methods, which is
| one of the reasons that Rcpp linkage has to be done the way that it is.? The
| exporting of symbols a as done by Matrix is restricted to C function entry
| points.

Ah, yes, that aspect too. Meant to mention, thanks for catching it.

In short: no beans. ?You may have to build one larger package.

Dirk

--
Gauss once played himself in a zero-sum game and won $50.
? ? ? ? ? ? ? ? ? ? ?-- #11 at http://www.gaussfacts.com

| Thanks for both your replies. I got it working the way I wanted. If
| someone's interested, a simple example with two packages can be
| downloaded here: http://www.ucl.ac.uk/~uctpjyy/downloads/RcppEx.zip
| 
| QuadFunc defines a C++ class that is used in C++ code in QuadFuncVec.

So you define function CxxFlags() and LdFlags() in your package just like we
do for Rcpp --- that is indeed just about the only way I could think of.

Glad know you got that working. 

Dirk

PS Also try 
   
   -e "QuadFunc:::CxxFlags()")

   instead of 

   -e "library(QuadFunc); QuadFunc:::CxxFlags()")

   as the ::: finds unexported symbols, that usually works from unloaded
   packages too.  And because of LinkingTo: QuadFunc you may not need the
   CxxFlags.

Gauss once played himself in a zero-sum game and won $50.
                      -- #11 at http://www.gaussfacts.com

Hi Dirk,

thanks for your reply.

PS Also try

  -e "QuadFunc:::CxxFlags()")

  instead of

  -e "library(QuadFunc); QuadFunc:::CxxFlags()")

  as the ::: finds unexported symbols, that usually works from unloaded
  packages too.

It works for unloaded packages that don't use modules. Since QuadFunc
uses modules, this results in an error

Rscript -e "QuadFunc:::CxxFlags()"

Error: .onLoad failed in loadNamespace() for 'QuadFunc', details:
  call: value[[3L]](cond)
  error: failed to load module QuadraticFunctionModule from package QuadFunc
Execution halted

whereas loading the library first doesn't

Rscript -e "library(QuadFunc); QuadFunc:::CxxFlags()"

Loading required package: methods
Loading required package: Rcpp
-IC:/R/win-library/2.12/QuadFunc/include

Perhaps I'm mis-using modules and should change something in my
NAMESPACE or in R/zzz.R to get rid of this error? You can get the same
error using Rcpp.package.skeleton( 'testmodA', module=TRUE ) (see the
first email).

And because of LinkingTo: QuadFunc you may not need the CxxFlags.

This is correct in this case. If another package links to QuadFuncVec,
then I think the dependencies become a bit more complicated (but I
might be mistaken here). Either the new package needs to add QuadFunc
to the LinkingTo: field, or QuadFuncVec:::CxxFlags() should return the
flags of all its dependencies. In this case I would choose to
implement QuadFuncVec:::CxxFlags() returning its own flags and those
from QuadFunc:::CxxFlags(). That way the user has to worry a bit less
about dependencies. At least, that was what I came up with after some
experimentation yesterday.

Are the LinkingTo: fields obtained for the current package only, or
also the LinkingTo: fields of the packages it links to also included?
E.g. packB has LinkingTo: packA, packC has LinkingTo: packB. Are the
include paths of packA automatically added to packC? That would make
the CxxFlags() unnecessary in this case.

Thanks,
Jelmer

Hi Jelmer,

| Hi Dirk,
| 
| thanks for your reply.
| 
| > PS Also try
| >
| >   -e "QuadFunc:::CxxFlags()")
| >
| >   instead of
| >
| >   -e "library(QuadFunc); QuadFunc:::CxxFlags()")
| >
| >   as the ::: finds unexported symbols, that usually works from unloaded
| >   packages too.
| 
| It works for unloaded packages that don't use modules. Since QuadFunc
| uses modules, this results in an error
| 
| > Rscript -e "QuadFunc:::CxxFlags()"
| Error: .onLoad failed in loadNamespace() for 'QuadFunc', details:
|   call: value[[3L]](cond)
|   error: failed to load module QuadraticFunctionModule from package QuadFunc
| Execution halted
| 
| whereas loading the library first doesn't
| 
| > Rscript -e "library(QuadFunc); QuadFunc:::CxxFlags()"
| Loading required package: methods
| Loading required package: Rcpp
| -IC:/R/win-library/2.12/QuadFunc/include

I see, and I stand corrected. Using modules may have that requirement as a
side-effect.
| 
| Perhaps I'm mis-using modules and should change something in my
| NAMESPACE or in R/zzz.R to get rid of this error? You can get the same
| error using Rcpp.package.skeleton( 'testmodA', module=TRUE ) (see the
| first email).
| 
| > And because of LinkingTo: QuadFunc you may not need the CxxFlags.
| 
| This is correct in this case. If another package links to QuadFuncVec,
| then I think the dependencies become a bit more complicated (but I
| might be mistaken here). Either the new package needs to add QuadFunc
| to the LinkingTo: field, or QuadFuncVec:::CxxFlags() should return the
| flags of all its dependencies. In this case I would choose to
| implement QuadFuncVec:::CxxFlags() returning its own flags and those
| from QuadFunc:::CxxFlags(). That way the user has to worry a bit less
| about dependencies. At least, that was what I came up with after some
| experimentation yesterday.

It should does not hurt. But for the compile step, you should only need the
header files, and R itself can point to those via LinkingTo.

Having a CxxFlags() function allows you to set other flags (maybe
-Dsomething) if you need it.  Nothing wrong with that -- I was merely trying
to point that you could make your life a little simpler.
 
| Are the LinkingTo: fields obtained for the current package only, or
| also the LinkingTo: fields of the packages it links to also included?
| E.g. packB has LinkingTo: packA, packC has LinkingTo: packB. Are the
| include paths of packA automatically added to packC? That would make
| the CxxFlags() unnecessary in this case.

I don't believe the DESCRIPTION files of dependent packages are read. So I
would just do this in DESCRIPTION for packC:

    LinkingTo: packA, packB

Does that help?

Dirk

| 
| Thanks,
| Jelmer

Gauss once played himself in a zero-sum game and won $50.
                      -- #11 at http://www.gaussfacts.com

Hi Dirk,

yes, this makes sense, thanks. I originally added the CxxFlags(),
because one of the packages I'm linking to defines where to find the
Boost headers, but I see that it's not needed in all cases.

Thanks!
Jelmer

Hi Jelmer,

On 16 May 2011 at 14:00, Jelmer Ypma wrote:
| Hi Dirk,
|
| thanks for your reply.
|
| > PS Also try
| >
| > ? -e "QuadFunc:::CxxFlags()")
| >
| > ? instead of
| >
| > ? -e "library(QuadFunc); QuadFunc:::CxxFlags()")
| >
| > ? as the ::: finds unexported symbols, that usually works from unloaded
| > ? packages too.
|
| It works for unloaded packages that don't use modules. Since QuadFunc
| uses modules, this results in an error
|
| > Rscript -e "QuadFunc:::CxxFlags()"
| Error: .onLoad failed in loadNamespace() for 'QuadFunc', details:
| ? call: value[[3L]](cond)
| ? error: failed to load module QuadraticFunctionModule from package QuadFunc
| Execution halted
|
| whereas loading the library first doesn't
|
| > Rscript -e "library(QuadFunc); QuadFunc:::CxxFlags()"
| Loading required package: methods
| Loading required package: Rcpp
| -IC:/R/win-library/2.12/QuadFunc/include

I see, and I stand corrected. Using modules may have that requirement as a
side-effect.
|
| Perhaps I'm mis-using modules and should change something in my
| NAMESPACE or in R/zzz.R to get rid of this error? You can get the same
| error using Rcpp.package.skeleton( 'testmodA', module=TRUE ) (see the
| first email).
|
| > And because of LinkingTo: QuadFunc you may not need the CxxFlags.
|
| This is correct in this case. If another package links to QuadFuncVec,
| then I think the dependencies become a bit more complicated (but I
| might be mistaken here). Either the new package needs to add QuadFunc
| to the LinkingTo: field, or QuadFuncVec:::CxxFlags() should return the
| flags of all its dependencies. In this case I would choose to
| implement QuadFuncVec:::CxxFlags() returning its own flags and those
| from QuadFunc:::CxxFlags(). That way the user has to worry a bit less
| about dependencies. At least, that was what I came up with after some
| experimentation yesterday.

It should does not hurt. But for the compile step, you should only need the
header files, and R itself can point to those via LinkingTo.

Having a CxxFlags() function allows you to set other flags (maybe
-Dsomething) if you need it. ?Nothing wrong with that -- I was merely trying
to point that you could make your life a little simpler.

| Are the LinkingTo: fields obtained for the current package only, or
| also the LinkingTo: fields of the packages it links to also included?
| E.g. packB has LinkingTo: packA, packC has LinkingTo: packB. Are the
| include paths of packA automatically added to packC? That would make
| the CxxFlags() unnecessary in this case.

I don't believe the DESCRIPTION files of dependent packages are read. So I
would just do this in DESCRIPTION for packC:

? ?LinkingTo: packA, packB

Does that help?

Dirk

|
| Thanks,
| Jelmer

--
Gauss once played himself in a zero-sum game and won $50.
? ? ? ? ? ? ? ? ? ? ?-- #11 at http://www.gaussfacts.com

Hi Jelmer,

| yes, this makes sense, thanks. I originally added the CxxFlags(),
| because one of the packages I'm linking to defines where to find the
| Boost headers, but I see that it's not needed in all cases.

Yes, sure, there can be many reasons to have PKG_CXXFLAGS. External
dependencies may be one.

But with your correct LinkingTo: usage, you do not need to set your own include
files via a CxxFlags() function.  

Glad you have it all working.

Cheers, Dirk

Gauss once played himself in a zero-sum game and won $50.
                      -- #11 at http://www.gaussfacts.com