[Rcpp-devel] Shared code in C++ files

Hi Martin -

I, too, am unsure about the mechanics of what you are wanting to do.  But
this piece:

 I can think of many applications in which the useR could supply a minimal

amount of C++ code (e.g., a log-likelihood function) to hook in with a
large amount of code provided by the developer in order to speed things up
considerably.

looks similar to what I am doing in a package that I maintain:  The user
writes some C++ code, compiles it via `R CMD SHLIB` and loads the shared
object into the current R session with `dyn.load()` (actually, I provide an
R function or ten to help manage that).  This is that "minimal amount of
C++ code" you were referring to.  The user then hands off  the address of
those symbols (`?getNativeSymbolInfo`) to the developer-provided code base
(in an R package), turn that into pointer to the function(s) that can be
called by the code-base as needed.  My package is written with Rcpp and
regular old C++;  all of the data that gets handed off to the user-defined
functions are C++ data types so I don't need to link back to Rcpp.  But
it's easy enough to link up with Rcpp  (or whatever else) when compiling
the shared object.

I'm pretty sure Dirk has similar(?) setup in his RcppDE package where
users can provide a compiled function for DEoptim to call when needed.
deSolve is another package that does something like this, but no Rcpp
involvement.

Of course, this requires some coordination between what is going on in the
shared object and the main code base, but it's not unreasonably complicated
to work out.  The flexibility you get is well worth it.   And absolutely no
problem juggling multiple user-defined functions in the same R session.

Best Regards,

Kyle Baron
Metrum Research Group
https://github.com/metrumresearchgroup/mrgsolve


On Thu, Apr 14, 2016 at 10:55 AM, JJ Allaire <jj.allaire at gmail.com> wrote:

I don't know enough about the mechanics of your scenario to comment
intelligently, but I would in general agree with Dirk that if there is a
way to put something like this in a package that's the way it ought to be
done. You could certainly dispatch to different C++ functions within a
package using a variety of mechanisms including C++ templates, C-style
function pointers, etc. (the mechanics of doing so for your situation I'm
unsure of, but it should be possible).

On Thu, Apr 14, 2016 at 12:25 AM, Martin Lysy <mlysy at uwaterloo.ca> wrote:

Hello Dirk,

For my specific purpose I have some reservations about package
creation.  Here is a description of the project I have in mind.

I would like to use Rcpp to create a set of generic MCMC algorithms,
into which a useR could easily hook their own target distributions written
in C++ with minimal effort.  For example, I would provide the following
files:

--------------------------------------
GibbsSampler.cpp

//[[Rcpp::export]]
NumericMatrix GibbsSampler(int nIter, NumericVector xInit, List
tuningParams) {
  // run some flavor of the Gibbs sampler
}
---------------------------------------
GibbsSampler.h

double logDensity(NumericVector x);
---------------------------------------

Then the useR would write MyDensityA.cpp, which contains the definition
of logDensity, compile the three files, and have a Gibbs sampler for their
specific density function written in C++ and ready to go in R.  However, a
useR might wish to use the GibbsSampler for a different density tomorrow.
They would have a different definition of logDensity in MyDensityB.cpp.
Ideally, the useR would have access to Gibbs samplers for both densities in
the same R session.

I can think of two ways of doing this with Rcpp:

1.  Compile with sourceCpp (this is what I'm currently doing).  There's
the minor issue of giving separate R names to each Gibbs sampler, but there
are many ways around that.  The major issue is that sourceCpp only accepts
a single .cpp file (or at least as far as my attempts were unsuccessful in
the original post).  So I'm stuck text-processing a bunch of .cpp and .h
files together (the actual project I'm working on has about a dozen of
each).

2.  Compile an entire R package for each of MyDensityA and MyDensityB.
However, it seems somewhat cumbersome to have a package loaded for every
density function in the workspace.  Moreover, naming conflicts are a bit
more tricky.  Right now (with sourceCpp), I'm using an interface of the form

smpA <- gibbs.sampler(density = MyDensityA, niter = 1e4)
smpB <- gibbs.sampler(density = MyDensityB, niter = 1e4)

This is to align with things like lm(formula = MyModel).  However, I
can't quite see how to do this with separate packages loaded.  Rather it
seems I'd need something like

smpA <- gibbs.sampler.MyDensityA(niter = 1e4)
smpB <- gibbs.sampler.MyDensityB(niter = 1e4)

However, to do this with packages I feel like I would still have to do
some text replacement, which I'm already doing with the sourceCpp approach.

In summary, I am not opposed to package creation, but I hope you can see
my reservations at taking this route.  Perhaps you could please suggest a
way to achieve what I'm after with separate Rcpp packages for each density
function.  I take it from your reluctance to answer my original post that
Rcpp only supports compilation of multiple files through the package
creation protocol.  I can think of many applications in which the useR
could supply a minimal amount of C++ code (e.g., a log-likelihood function)
to hook in with a large amount of code provided by the developer in order
to speed things up considerably.  So in my opinion it would be worthwhile
to devise a mechanism to do this correctly.

Best regards,


Martin Lysy
Assistant Professor of Statistics
Department of Statistics and Actuarial Science
University of Waterloo

On Wed, Apr 13, 2016 at 5:34 PM, Dirk Eddelbuettel <edd at debian.org>
wrote:

Martin,

Please please please look into creating a package.

If you use RStudio:  File -> New Project -> (New or Existing) Directory
->
Package and then select Rcpp.

If not, consider install the (very tiny) pkgKitten package and call
Rcpp.package.skeleton() from Rcpp itself (but enhanced by pkgKitten if
present) for a saner package.

Cheers, Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

--
Kyle Baron
Metrum Research Group
860-735-7043, Ext. 202
kyleb at metrumrg.com