[R-pkg-devel] Convention or standards for using header library (e.g. Eigen)
On 24.06.2023 19:44, Dirk Eddelbuettel wrote:
On 24 June 2023 at 21:35, Stephen Wade wrote: | Doesnt seem like the system package is worth it. Should the convention | simply be to bundle the headers in the package then? What about package | size - is there some limit to the size of included libraries/headers to | consider for CRAN? Here is one (drastic) example: $ du -csh /usr/local/lib/R/site-library/BH 156M /usr/local/lib/R/site-library/BH 156M total $
Of course one should always try to keep software as small as possible and not waste space. For binary packages, we are aware that some packages are large for the reason Dirk explains. CRAN typically does not complain here, although there are cases where we have to consider if it makes sense to distribute that huge software is system libraries may be available. The size restriction that applies for CRAN packages is the 5MB threshold for the source package size. Best, Uwe Ligges
Note that the package was smaller when it started (in 2013). (Note that the last time I checked its size, the largest (not just headers) package I know of on CRAN still was about twice as large still.) Anyway: as you are starting to see, this is a somewhat complex problem. Header packages are one approach. _Writing R Extensions_ mentions pure header packages and name-checks my packages BH, RcppArmadillo and RcppEigen in Section 1.1.3. I once wrote a short paper on this [1] (also a vignette [2]) where I more or less recommend header packages because compiled ones are so much harder. Recognise for example that a) no cross-OS way to check for packages exists (though pkg-config comes close), b) no general package managers exist, c) configure and cmake come close (but cmake is also an added system requirement; and configure is a no-show on Windows) and d) even within a given OS and release you may have very different versions. Lastly also: e) some packages (RcppEigen is an example) have patches the system library would not have applied (!!). So to me a simplified view is that just as R "abstracts away" POSIX so that we can always say e.g. 'dir.exists(path)' no matter where R runs, having a package with headers ensure we get a consistent _and reliable_ compilation experience from client packages. This matters. Now, there are clearly downsides. With my Debian maintainer hat on, I have to defend including Armadillo withon RcppArmadillo because the distro has it too (but then version skew ie d) above and ease of use and consistency etc dominate so we continue to ship RcppArmadillo). At the same time, at CRAN we have needless duplications. For example, my RcppCCTZ package was the first to offer the nice (Google made but not a Google 'product') CCTZ library for R use (starting in 2015). But when I last checked a year or so ago, four other packages now included redundant extra copies. Also happens with Eigen. Not great. On the other side, packages with full (included or not) libraries work too, but they are more effort to portably provide them, to explain to users where to get them and keep them current and so. It is hard (or even impossible) for R to fill in as a _general system_ package manager across all OSs and deployments. There is a new kid on this block [3] we are starting to use at work, and which may help in time across the platforms that R uses. To be seen... So to sum up: I think header packages are great, and I maintain a few, both large and small in size. I would encourage you to try them. For RcppEigen, you can just use LinkingTo: to gets its headers. Some 400+ packages rely on it. (And its over 1000 for Armadillo now, and over 300 for BH.) Hth, Dirk [1] https://arxiv.org/abs/1911.06416 [2] https://cran.r-project.org/web/packages/Rcpp/vignettes/Rcpp-libraries.pdf [3] https://vcpkg.io/en/