hi all, i posted a question before about this, but i may have been too
cryptic to understand.
in short, there exists an R package that someone is writing. this
package depends on a custom library (written in C,), compiled as a
shared, and called by the package's functions via the .Call(...)
method.
we are testing out different code implementations of the compiled
library functions, and thus have distinct compiled shared object files
(.so files in UNIX-like systems). currently this works just fine via
the R command prompt using the dyn.load(...) and dyn.unload(...)
functions. consider the following scenario:
two libraries of the same functions (different implementations) exist
at a user-level directory (we don't have system /lib or /usr/lib
access). these two files reside here:
~/lib/mylib.v1.so and ~/lib/mylib.v2.so
mylib.so is a copy (or link) to mylib.v1.so
start R, load the library via 'dyn.load("~/lib/mylib.so")'
execute R function that uses .Call(...) to a compiled function.
unload the library via 'dyn.unload("~/lib/mylib.so")'
copy mylib.v2.so to mylib.so
load the library via 'dyn.load("~/lib/mylib.so")'
execute R function again and compare results
(note that i know one could just load the specific versions of the
library in the above scenario, but it helps with consistency for the
following case.)
now, i'd like to have the R package written such that it expects the
library mylib.so, without having to load it explicitly via the dyn.load
(...) command. the package has a NAMESPACE, so i've tried this
solution by including (in zzz.R) this function:
.onLoad <- function(libname, package) library.dynam("mylib")
however, this (and variants of it i've tried) doesn't seem to work as
R will only search for the shared objects in the R package directories
themselves (particularly the /libs/ sub-directories of each package,
or the one you specify). my .so files sit external to an R package
directory, and also external to UNIX system-level libs.
(yes, i've set my LD_LIBRARY_PATH shell environment variable to
include ~/lib/, by the way.)
i've also tried the useDynLib(...) directive in the package NAMESPACE
file, but to no avail, as this seems to mimic library.dynam and only
search *inside* package contents for shared object libraries.
so, simply put:
i have a shared object library (compiled by R CMD SHLIB, to ensure
proper linking), in a user-level *local* lib directory (~/lib). this
directory is on the LD_LIBRARY_PATH (as it is used to test
replacements for some system-level libraries).
i would like to have an R package that dynamically loads this shared
object library upon package loading (or attachment, if needed), as
functions in the package call methods in the library via the .Call
(...) procedure.
i'm happy always having my shared object library have a single name (i
can update the current one when switching between versions), and doing
so would allow for a single line in the R package code to dynamically
load the (whichever version is current) library.
problem is, i haven't been able to do this yet.
my workaround is to constantly swap into myRPackage/libs/ directory a
version of the shared object library called myRPackage.so, and load it
via the useDynLib(...) directive in the NAMESPACE file of the
package. but this is cumbersome and doesn't allow other people to
have a single version of the R package to use as part of a team effort
to debug/test both the package's R code and my C code for the library
functions.
hopefully this makes a bit more sense, and if anyone has any tips on
how R actually loads shared objects it would be greatly appreciated.
so far all i can tell is that R will only look inside R packages in
the /libs/ directory, or at system-wide /lib/, /usr/lib/, etc.
directories for shared object files. (unless one explicitly uses
dyn.load with a hard-coded path, in which case it loads the library
just fine, but this is not an elegant solution.)
thanks very much for any help!
-m
shared object location
4 messages · Murat Tasan, Seth Falcon
Hi,
On 1/29/10 12:58 PM, Murat Tasan wrote:
problem is, i haven't been able to do this yet. my workaround is to constantly swap into myRPackage/libs/ directory a version of the shared object library called myRPackage.so, and load it via the useDynLib(...) directive in the NAMESPACE file of the package. but this is cumbersome and doesn't allow other people to have a single version of the R package to use as part of a team effort to debug/test both the package's R code and my C code for the library functions. hopefully this makes a bit more sense, and if anyone has any tips on how R actually loads shared objects it would be greatly appreciated.
I'm afraid I don't have a solution for you. Reading through your post, I'm confused about why you want to have your .so file outside of the package? Why not follow the usual scheme and R CMD INSTALL yourPkg with the version of R code and C code you want to use/test. This would best allow other people to have a single version of the R package, no? + seth
Seth Falcon | @sfalcon | http://userprimary.net/user
1 day later
The primary reason is for development. I do install via the traditional R CMD INSTALL (or variant) method, but I want to keep the C code external from the R package. In particular, I want to be able to modify the C code (and thus the compiled .so library functions) without having to constantly re- install the R package, or constantly copy the shared object library to the /libs/ subdirectory in the R package directory. The crux of the matter is that this seems do-able: it basically works using dyn.load, just not easily with library.dynam, which is what confuses me. The vast majority of other UNIX-ish applications will somewhere allow me to specify a non-standard library location, if needed In many cases, this is done (in an ad-hoc, and thus unstable manner) using LD_LIBRARY_PATH, but this doesn't seem to be working for me in this instance.
On Jan 29, 4:53?pm, Seth Falcon <s... at userprimary.net> wrote:
Hi, On 1/29/10 12:58 PM,MuratTasan wrote:
problem is, i haven't been able to do this yet.
my workaround is to constantly swap into myRPackage/libs/ directory a version of the shared object library called myRPackage.so, and load it via the useDynLib(...) directive in the NAMESPACE file of the package. ?but this is cumbersome and doesn't allow other people to have a single version of the R package to use as part of a team effort to debug/test both the package's R code and my C code for the library functions.
hopefully this makes a bit more sense, and if anyone has any tips on how R actually loads shared objects it would be greatly appreciated.
I'm afraid I don't have a solution for you. ?Reading through your post, I'm confused about why you want to have your .so file outside of the package? Why not follow the usual scheme and R CMD INSTALL yourPkg with the version of R code and C code you want to use/test. ?This would best allow other people to have a single version of the R package, no? + seth -- Seth Falcon | @sfalcon |http://userprimary.net/user
______________________________________________ R-h... at r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
as a motivating example, consider the case where multiple/many R
packages are being developed that all use the same shared object
functions.
each time (during development) that the shared object file changes we
could go alter every single R package directory, or simply have the R
package utilize the most recent single copy of the shared object's
functions.
this is essentially what happens already with most programs and the
common libraries in /usr/lib, /usr/local/lib, etc.
but during development of the library, it's preferable to keep this
library out of those system-level trees, and instead somewhere more
local (e.g. in the developer's home directory, or a location set aside
by a team of developers, e.g. /dev/lib).
then i would like for each of the R packages to do something like
'library.dynam("testlib1")', looking along the LD_LIBRARY_PATH for the
R session (which will include /dev/lib), for the library /dev/lib/
testlib1.so.
but it appears library.dynam can ONLY search inside package
contents... is this correct?
i can keep telling every package writer to include 'dyn.load("/dev/lib/
testlib1.so")' somewhere in their package, but this is fragile (as
dyn.load requires an exact path to be specified, instead of using
LD_LIBRARY_PATH).
so it appears i either have fragility, or an error-prone process of
updating the .so files in each R package separately each time the
library changes (which is frequently).
i have to imagine there is some method in R that i just cannot
identify that will allow for linking/loading of shared objects along
the LD_LIBRARY_PATH.
On Jan 30, 5:52?pm, Murat Tasan <mmu... at gmail.com> wrote:
The primary reason is for development. I do install via the traditional R CMD INSTALL (or variant) method, but I want to keep the C code external from the R package. In particular, I want to be able to modify the C code (and thus the compiled .so library functions) without having to constantly re- install the R package, or constantly copy the shared object library to the /libs/ subdirectory in the R package directory. The crux of the matter is that this seems do-able: it basically works using dyn.load, just not easily with library.dynam, which is what confuses me. The vast majority of other UNIX-ish applications will somewhere allow me to specify a non-standard library location, if needed In many cases, this is done (in an ad-hoc, and thus unstable manner) using LD_LIBRARY_PATH, but this doesn't seem to be working for me in this instance. On Jan 29, 4:53?pm, Seth Falcon <s... at userprimary.net> wrote:
Hi,
On 1/29/10 12:58 PM,MuratTasan wrote:
problem is, i haven't been able to do this yet.
my workaround is to constantly swap into myRPackage/libs/ directory a version of the shared object library called myRPackage.so, and load it via the useDynLib(...) directive in the NAMESPACE file of the package. ?but this is cumbersome and doesn't allow other people to have a single version of the R package to use as part of a team effort to debug/test both the package's R code and my C code for the library functions.
hopefully this makes a bit more sense, and if anyone has any tips on how R actually loads shared objects it would be greatly appreciated.
I'm afraid I don't have a solution for you. ?Reading through your post, I'm confused about why you want to have your .so file outside of the package?
Why not follow the usual scheme and R CMD INSTALL yourPkg with the version of R code and C code you want to use/test. ?This would best allow other people to have a single version of the R package, no?
+ seth
-- Seth Falcon | @sfalcon |http://userprimary.net/user
______________________________________________ R-h... at r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
______________________________________________ R-h... at r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.