Skip to content

include dll in R-package

15 messages · Dirk Eddelbuettel, Kasper Daniel Hansen, Jeff Ryan +4 more

#
Hi,

We have several projects in the center done by researchers over years
in Fortran, there are copy right issues etc to prevent us from
giving away the source codes, but a lot of social scientist are 
interested to use the program. We tried to use dlls to make plugins 
(available in our website) in various statistics platforms 
(SAS, STATA and R) to make it available to general public.

We used to be able to build R package using dll on version R-2.7.0,
, but failed to build with newer R version with the same source. 
Any help is greatly appreciated. 

When I try to build R package for newer version of R, I could build 
to get "lcca_1.0.0.tar.gz", but when I "R CMD check lcca_1.0.0.tar.gz", 
I got error message as:

* using log directory 'D:/project/LCCA/lcca.Rcheck'
* using R version 2.15.1 (2012-06-22)
* using platform: i386-pc-mingw32 (32-bit)
* using session charset: ISO8859-1
* checking for file 'lcca/DESCRIPTION' ... OK
* checking extension type ... Package
* this is package 'lcca' version '1.0.0'
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... WARNING
Found the following executable file(s):
  libs/lcca.dll
Source packages should not contain undeclared executable files.
See section 'Package structure' in the 'Writing R Extensions' manual.
* checking whether package 'lcca' can be installed ... ERROR
Installation failed.
See 'D:/project/LCCA/lcca.Rcheck/00install.out' for details.
 
and the message in 00install.out is:

* installing *source* package 'lcca' ...
** R
** data
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error in library.dynam(lib, package, package.lib) : 
  DLL 'lcca' not found: maybe not installed for this architecture?
Error: loading failed
Execution halted
ERROR: loading failed
* removing 'D:/project/LCCA/lcca.Rcheck/lcca'


Thanks a lot!
Liy
#
On 24 August 2012 at 09:06, LIYING HUANG wrote:
| We have several projects in the center done by researchers over years
| in Fortran, there are copy right issues etc to prevent us from
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| giving away the source codes, but a lot of social scientist are 
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| interested to use the program. We tried to use dlls to make plugins 
| (available in our website) in various statistics platforms 
| (SAS, STATA and R) to make it available to general public.
| 
| We used to be able to build R package using dll on version R-2.7.0,
| , but failed to build with newer R version with the same source. 
| Any help is greatly appreciated. 
| 
| When I try to build R package for newer version of R, I could build 
| to get "lcca_1.0.0.tar.gz", but when I "R CMD check lcca_1.0.0.tar.gz", 
| I got error message as:
| 
| * using log directory 'D:/project/LCCA/lcca.Rcheck'
| * using R version 2.15.1 (2012-06-22)
| * using platform: i386-pc-mingw32 (32-bit)
| * using session charset: ISO8859-1
| * checking for file 'lcca/DESCRIPTION' ... OK
| * checking extension type ... Package
| * this is package 'lcca' version '1.0.0'
| * checking package namespace information ... OK
| * checking package dependencies ... OK
| * checking if this is a source package ... OK
| * checking if there is a namespace ... OK
| * checking for executable files ... WARNING
| Found the following executable file(s):
|   libs/lcca.dll
| Source packages should not contain undeclared executable files.
| See section 'Package structure' in the 'Writing R Extensions' manual.
| * checking whether package 'lcca' can be installed ... ERROR

This tells you that in order to have a proper package, you need to include
the very source code you want to hide.

This is a CRAN Policy decision enforced by current R versions (but not the
rather old version you compared against), and there is now way around it.
You could try to construct "defunct" packages lacking the DLLs and instruct
the users to get them from somewhere else, but that is at the same rather
error prone (as you will lack all the built-time checks you would have with
source code, as well as a better assurrance that compatible tools are used)
and distasteful as CRAN is about Open Source.

So your best bet may be to go back to the copyright holders....

Dirk
#
Any package can link to external DLLs.  You'll need to make that an
explicit dependency, have some configure script and distribute the
library somewhere else, but many packages on CRAN already do this.

My RBerkeley uses a user installed Oracle Berkeley DB that is linked
to at configure time, of course there are many other examples as well.

And distributing _off_ CRAN is of course a reasonable alternative.

HTH
Jeff

On Fri, Aug 24, 2012 at 9:21 AM, Kasper Daniel Hansen
<kasperdanielhansen at gmail.com> wrote:

  
    
#
Thanks for the clarification. Just wonder if I don't host the 
package on CRAN, am I able to build a package without
source codes to distribute it from my institutions homepage?
Seems not for the newer version?

Thanks!
Liying


----- Original Message -----
From: "Kasper Daniel Hansen" <kasperdanielhansen at gmail.com>
To: "Dirk Eddelbuettel" <edd at debian.org>
Cc: "LIYING HUANG" <lxh37 at psu.edu>, r-devel at r-project.org
Sent: Friday, August 24, 2012 10:21:18 AM
Subject: Re: [Rd] include dll in R-package
On Fri, Aug 24, 2012 at 10:16 AM, Dirk Eddelbuettel < edd at debian.org > wrote:

        
On 24 August 2012 at 09:06, LIYING HUANG wrote:
| We have several projects in the center done by researchers over years 
| in Fortran, there are copy right issues etc to prevent us from 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

| giving away the source codes, but a lot of social scientist are 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 


| interested to use the program. We tried to use dlls to make plugins 
| (available in our website) in various statistics platforms 
| (SAS, STATA and R) to make it available to general public. 
| 
| We used to be able to build R package using dll on version R-2.7.0, 
| , but failed to build with newer R version with the same source. 
| Any help is greatly appreciated. 
| 
| When I try to build R package for newer version of R, I could build 
| to get "lcca_1.0.0.tar.gz", but when I "R CMD check lcca_1.0.0.tar.gz", 
| I got error message as: 
| 
| * using log directory 'D:/project/LCCA/lcca.Rcheck' 
| * using R version 2.15.1 (2012-06-22) 
| * using platform: i386-pc-mingw32 (32-bit) 
| * using session charset: ISO8859-1 
| * checking for file 'lcca/DESCRIPTION' ... OK 
| * checking extension type ... Package 
| * this is package 'lcca' version '1.0.0' 
| * checking package namespace information ... OK 
| * checking package dependencies ... OK 
| * checking if this is a source package ... OK 
| * checking if there is a namespace ... OK 
| * checking for executable files ... WARNING 
| Found the following executable file(s): 
| libs/lcca.dll 
| Source packages should not contain undeclared executable files. 
| See section 'Package structure' in the 'Writing R Extensions' manual. 
| * checking whether package 'lcca' can be installed ... ERROR 

This tells you that in order to have a proper package, you need to include 
the very source code you want to hide. 

This is a CRAN Policy decision enforced by current R versions (but not the 
rather old version you compared against), and there is now way around it. 
You could try to construct "defunct" packages lacking the DLLs and instruct 
the users to get them from somewhere else, but that is at the same rather 
error prone (as you will lack all the built-time checks you would have with 
source code, as well as a better assurrance that compatible tools are used) 
and distasteful as CRAN is about Open Source. 

So your best bet may be to go back to the copyright holders.... 



Dirk's comments are extremely relevant if you were hoping to host the package on CRAN (which you basically won't be allowed to). You can still distribute it from your institutions homepage I think. 


Kasper
1 day later
#
On Fri, Aug 24, 2012 at 7:39 AM, LIYING HUANG <lxh37 at psu.edu> wrote:
I think that is a valid point; R is not CRAN and CRAN is not R, but
for obvious reasons they sometimes tend to blend.

Either way, I wanna second what's already been said by others.  Try
your best to distribute as much as possible of your tools as open
source.  If there are pieces that are impossible to put under open
source, at least identify them and maybe put them in a separate
package, and make the rest open source.  On our end, doing science
with non open source/access tools is really a pain, and I know lots of
people avoiding such tools as far as possible, because you know that
one day or the other you'll pay the price of not being able to
check/validate what's really going on in your analysis.  So open
source is great...

...and to make the point, have a look at
https://svn.r-project.org/R/trunk/src/library/tools/R/check.R and
you'll see that you most likely will be able to disable those tests.
Alternatively, make sure to read 'Checking packages' in Writing R
Extensions.  It may be dense, but it is very informative and useful.
One section reads "Use R CMD check --help to obtain more information
about the usage of the R package checker. A subset of the checking
steps can be selected by adding command-line options. It also allows
customization by setting environment variables _R_CHECK_*_:, as
described in Tools: [...]."  So, check out that 'Tools' documentation,
and I think you'll find a solution.

My $.02 GFPL v1.3 comments

/Henrik
#
Jeff,

Thanks for the reply.

For "explicit dependency", can dll and functions be specified 
in namespace and imported by the package automatically? or
the user has to use dyn.load and is.loaded to load the dll
and functions in R? If it is the former, could you point
me to an example that I could learn? Thanks!

Liying

----- ???? -----
???: "Jeff Ryan" <jeff.a.ryan at gmail.com>
???: "Kasper Daniel Hansen" <kasperdanielhansen at gmail.com>
??: r-devel at r-project.org, "Dirk Eddelbuettel" <edd at debian.org>
????: ???, 2012? 8 ? 24? ?? 10:28:50
??: Re: [Rd] include dll in R-package

Any package can link to external DLLs.  You'll need to make that an
explicit dependency, have some configure script and distribute the
library somewhere else, but many packages on CRAN already do this.

My RBerkeley uses a user installed Oracle Berkeley DB that is linked
to at configure time, of course there are many other examples as well.

And distributing _off_ CRAN is of course a reasonable alternative.

HTH
Jeff

On Fri, Aug 24, 2012 at 9:21 AM, Kasper Daniel Hansen
<kasperdanielhansen at gmail.com> wrote:

  
    
1 day later
#
I am quite new in R, I looked at R manual- how to build R Package, 
still not very clear. Yes, it would be very helpful if I could 
download a package having Fortran source codes to look at, Do 
you know any package built with Fortran source code? Thanks!

Liy


----- ???? -----
???: "Kasper Daniel Hansen" <kasperdanielhansen at gmail.com>
???: "LIYING HUANG" <lxh37 at psu.edu>
??: r-devel at r-project.org
????: ???, 2012? 8 ? 24? ?? 10:19:14
??: Re: [Rd] include dll in R-package

You need to setup a Makevars.win and at lest modify 
PKG_LIBS 
and probably 
PKG_FFLAGS 
PKG_CFLAGS 
depending on how you link to the DLL. 


I suggest downloading example packages from Bioc / CRAN to look at. 


Kasper
On Fri, Aug 24, 2012 at 9:06 AM, LIYING HUANG < lxh37 at psu.edu > wrote:
Hi, 

We have several projects in the center done by researchers over years 
in Fortran, there are copy right issues etc to prevent us from 
giving away the source codes, but a lot of social scientist are 
interested to use the program. We tried to use dlls to make plugins 
(available in our website) in various statistics platforms 
(SAS, STATA and R) to make it available to general public. 

We used to be able to build R package using dll on version R-2.7.0, 
, but failed to build with newer R version with the same source. 
Any help is greatly appreciated. 

When I try to build R package for newer version of R, I could build 
to get "lcca_1.0.0.tar.gz", but when I "R CMD check lcca_1.0.0.tar.gz", 
I got error message as: 

* using log directory 'D:/project/LCCA/lcca.Rcheck' 
* using R version 2.15.1 (2012-06-22) 
* using platform: i386-pc-mingw32 (32-bit) 
* using session charset: ISO8859-1 
* checking for file 'lcca/DESCRIPTION' ... OK 
* checking extension type ... Package 
* this is package 'lcca' version '1.0.0' 
* checking package namespace information ... OK 
* checking package dependencies ... OK 
* checking if this is a source package ... OK 
* checking if there is a namespace ... OK 
* checking for executable files ... WARNING 
Found the following executable file(s): 
libs/lcca.dll 
Source packages should not contain undeclared executable files. 
See section 'Package structure' in the 'Writing R Extensions' manual. 
* checking whether package 'lcca' can be installed ... ERROR 
Installation failed. 
See 'D:/project/LCCA/lcca.Rcheck/00install.out' for details. 

and the message in 00install.out is: 

* installing *source* package 'lcca' ... 
** R 
** data 
** preparing package for lazy loading 
** help 
*** installing help indices 
** building package indices 
** testing if installed package can be loaded 
Error in library.dynam(lib, package, package.lib) : 
DLL 'lcca' not found: maybe not installed for this architecture? 
Error: loading failed 
Execution halted 
ERROR: loading failed 
* removing 'D:/project/LCCA/lcca.Rcheck/lcca' 


Thanks a lot! 
Liy 

______________________________________________ 
R-devel at r-project.org mailing list 
https://stat.ethz.ch/mailman/listinfo/r-devel
#
On 28-08-2012, at 01:29, LIYING HUANG wrote:

            
To mention a few:

deSolve
minpack.lm
nleqslv
rootSolve
xts

and probably many more.

Berend
#
Thanks a lot for those examples, is there any example with Fortran
source codes which could depend on an external Fortran DLL?

Liy

----- ???? -----
???: "Berend Hasselman" <bhh at xs4all.nl>
???: "LIYING HUANG" <lxh37 at psu.edu>
??: "Kasper Daniel Hansen" <kasperdanielhansen at gmail.com>, r-devel at r-project.org
????: ???, 2012? 8 ? 28? ?? 7:02:02
??: Re: [Rd] include dll in R-package
On 28-08-2012, at 01:29, LIYING HUANG wrote:

            
To mention a few:

deSolve
minpack.lm
nleqslv
rootSolve
xts

and probably many more.

Berend
#
On 29-08-2012, at 05:31, LIYING HUANG wrote:

            
I wouldn't know.

Berend
#
On 29/08/2012 12:19, Berend Hasselman wrote:
What does 'depend on' mean here?  (All the context has been removed, 
pace the posting guide.)

If it means 'link to', then any package which uses BLAS or LAPACK is an 
example.  But strictly you cannot link to a DLL (although you can on 
Windows, but on OS X what R calls DLL is not what OS X calls a 'dynamic 
library').

If you want to dynamically load an external DLL then

- there are no Fortran examples on CRAN
- you need to ensure that the DLL is already linked to the Fortran 
runtime (which typically Fortran DLLs are not).
1 day later
#
One very simple question, probably not appropriate to post this to rdevel list,
I put all Fortran files in src, when I do R CMD check, it tries to compile
the files by alphabetical order. I tried to modify Makevar, but no luck 
after staring at the manual for hours. It seems that I could use R CMD SHLIB 
as following to compile the codes first to get around this problem before 
doing "R CMD check", but I am curious how to specify the file compilation order 
in Makevar? Thanks!

R CMD SHLIB -o lcca.dll error_handler.f90 constants.f90

Liy

----- Original Message -----
From: "Berend Hasselman" <bhh at xs4all.nl>
To: "LIYING HUANG" <lxh37 at psu.edu>
Cc: "Kasper Daniel Hansen" <kasperdanielhansen at gmail.com>, r-devel at r-project.org
Sent: Tuesday, August 28, 2012 7:02:02 AM
Subject: Re: [Rd] include dll in R-package
On 28-08-2012, at 01:29, LIYING HUANG wrote:

            
To mention a few:

deSolve
minpack.lm
nleqslv
rootSolve
xts

and probably many more.

Berend
#
On 30/08/2012 16:59, LIYING HUANG wrote:
Well, it is in the manual you stared at ....  However, for Fortran the 
order of compilation never matters.   If you have Fortran90 modules, it 
may and that is why Makefiles can have dependencies.

1) You can set OBJECTS to set the order.

2) To avoid problems with parallel makes you should set dependencies, e.g.

constants.o: error_handler.o

or more likely some module file.

Beyond that, R-devel is indeed not the place for a tutorial on Makefiles 
and compilation.