Skip to content

[R-pkg-devel] R_registerRoutines, etc.

20 messages · Dirk Eddelbuettel, Avraham Adler, Duncan Murdoch +1 more

Original
Rolf Turner
# 
I have, like many others it would appear, been struggling with
the new-ish convention of requiring --- or quasi-requiring --- that 
"routines" be "registered" and the warning generated by R CMD check to 
the effect:
Well, the material in the "Writing R Extensions" manual is completely 
incomprehensible to the human mind, which is what I'm equipped with. 
However I found a posting by Ege Rubak on this topic which sent me by a 
slightly roundabout route to a posting by Dirk Eddelbuttel
which told me to use package_native_routine_registration_skeleton() from 
the tools package.

After a bit more struggle, I found that that did the trick.  I have, 
however, a couple of questions remaining.

(1) I found that having an R function with the same name as that of a
routine (Fortran subroutine in this case) that it called, causes all 
sorts of chaos.  I had a function "binsrt" that called a Fortran 
subroutine named "binsrt" and a function "mnnd" that called a Fortran 
subroutine named "mnnd".  This induced several fairly mysterious 
warnings.  I resolved the issue by renaming the R functions "binsrtR"
and "mnndR" respectively, so as to eliminate the name conflict.

Would this be the recommended procedure, or is there a cleverer way to 
eliminate the problem?

(2) The help for package_native_routine_registration_skeleton() says:
OTOH a post from Ege Rubak (answering a question like unto mine from 
another user) basically says "Don't bother.  It doesn't really matter."

However, being a Good Little Boy, I like to follow instructions exactly. 
  So I tried to replace the "void *" strings by the "actual types", but 
then all hell broke loose.  Consequently I went back to the "void *" 
structures.  That appears to work, but:

    (a) Are there any perils lurking if one just leaves "void *" as is,
        throughout?

    (b) For the sake of completeness, how *does* one replace the "void *"
        constructions with "actual types" in a correct manner?

         Example:  In my init.c file I currently have (as produced by
         package_native_routine_registration_skeleton()) :
The code in mnnd.f reads:
I.e. the "actual types are "double precision",
         "double precision", "integer", "double precision",
         "double precision".

         So in this case I should (?) replace
by .... what?  Can anyone tell me?

Thanks.

cheers,

Rolf Turner
Dirk Eddelbuettel
#
On 23 April 2017 at 09:25, Rolf Turner wrote:
| I have, like many others it would appear, been struggling with
| the new-ish convention of requiring --- or quasi-requiring --- that 
| "routines" be "registered" and the warning generated by R CMD check to 
| the effect:
| 
| > Found no calls to: 'R_registerRoutines', 'R_useDynamicSymbols'
| >
| > It is good practice to register native routines and to disable symbol
| > search.
| >
| > See 'Writing portable packages' in the 'Writing R Extensions' manual.
| 
| Well, the material in the "Writing R Extensions" manual is completely 
| incomprehensible to the human mind, which is what I'm equipped with. 
| However I found a posting by Ege Rubak on this topic which sent me by a 
| slightly roundabout route to a posting by Dirk Eddelbuttel

There is a transcribed Umlaut in there:  Eddelbuettel (ie 'ue' not 'u')

| which told me to use package_native_routine_registration_skeleton() from 
| the tools package.
| 
| After a bit more struggle, I found that that did the trick.  I have, 
| however, a couple of questions remaining.
| 
| (1) I found that having an R function with the same name as that of a
| routine (Fortran subroutine in this case) that it called, causes all 
| sorts of chaos.  I had a function "binsrt" that called a Fortran 
| subroutine named "binsrt" and a function "mnnd" that called a Fortran 
| subroutine named "mnnd".  This induced several fairly mysterious 
| warnings.  I resolved the issue by renaming the R functions "binsrtR"
| and "mnndR" respectively, so as to eliminate the name conflict.
| 
| Would this be the recommended procedure, or is there a cleverer way to 
| eliminate the problem?

That has hit me too, and I also found the error message less than perfectly
clear.  It all makes sense in the large sense -- we now register functions,
and R and compiled code (C++ for me, Fortran for you) "compete" for unique
identifiers.

The "official" way in Writing R Extensions fixes this by automagically
prefixing functions called via .Call() with a "c_" prefix.  You can turn that
on in the NAMESPACE file as further argument to useDynLib().  I am not (yet?)
doing this in all packages, and I "spied" on one recent update of a package
by an R Core member that didn;t either.  When in Rome ...
 
| (2) The help for package_native_routine_registration_skeleton() says:
| 
| > Optionally the skeleton will include declarations for the registered
| > routines: they should be checked against the C/Fortran source code, not
| > least as the number of arguments is taken from the R code. For .Call and
| > .External calls they will often suffice, but for .C and .Fortran calls
| > the void * arguments would ideally be replaced by the actual types.
| 
| OTOH a post from Ege Rubak (answering a question like unto mine from 
| another user) basically says "Don't bother.  It doesn't really matter."
| 
| However, being a Good Little Boy, I like to follow instructions exactly. 
|   So I tried to replace the "void *" strings by the "actual types", but 
| then all hell broke loose.  Consequently I went back to the "void *" 
| structures.  That appears to work, but:
| 
|     (a) Are there any perils lurking if one just leaves "void *" as is,
|         throughout?
| 
|     (b) For the sake of completeness, how *does* one replace the "void *"
|         constructions with "actual types" in a correct manner?
| 
|          Example:  In my init.c file I currently have (as produced by
|          package_native_routine_registration_skeleton()) :
| 
| > extern void F77_NAME(mnnd)(void *, void *, void *, void *, void *);
| 
|          The code in mnnd.f reads:
| 
| > subroutine mnnd(x,y,n,dminbig,dminav)
| > implicit double precision(a-h,o-z)
| > .....
| 
|          I.e. the "actual types are "double precision",
|          "double precision", "integer", "double precision",
|          "double precision".
| 
|          So in this case I should (?) replace
| 
| > extern void F77_NAME(mnnd)(void *, void *, void *, void *, void *);
| 
|          by .... what?  Can anyone tell me?

No idea. I don't deal in Fortran. C++ is more than enough fun for me ;-)

Dirk
Rolf Turner
#
On 23/04/17 09:44, Dirk Eddelbuettel wrote:
<SNIP>
Arrgggghhh!  Sorry 'bout that.  I'm usually fairly careful about getting 
people's names right, since I am myself annoyed by being called "Rolph" 
or "Ralph" or "Ralf" or "Ross" (!!!).

<SNIP>
The foregoing "official" way would seem to apply to functions called by 
".Call" (which I never use; it is way over my head).  What about 
functions called by ".Fortran()" or ".C()"?

<SNIP>
Well, C++ is too much for me.  I find Fortran generally very easy.  Be 
that as it were, are there any Fortran users out there who can answer my 
question, above?

Thanks Dirk.

cheers,

Rolf
Avraham Adler
#
On Sat, Apr 22, 2017 at 6:13 PM, Rolf Turner <r.turner at auckland.ac.nz> wrote:
I cannot answer your question completely, as I use .Call() to call
Fortran from R via C instead of using .Fortran?from what I have read
and researched, it is faster.

That being said, maybe this will help. Given a subroutine in Fortran
which starts like:

    subroutine ddelap_f(x, nx, a, na, b, nb, l, nl, lg, pmfv) bind(C,
name="ddelap_f")

    integer(kind = c_int), intent(in), value         :: nx, na, nb, nl
    ! Sizes
    real(kind = c_double), intent(in), dimension(nx) :: x
    ! Observations
    real(kind = c_double), intent(out), dimension(nx):: pmfv
    ! Result
    real(kind = c_double), intent(in)                :: a(na), b(nb),
l(nl)! Parameters
    integer(kind = c_int), intent(in)                :: lg
    ! Log flag
    integer                                          :: i
    ! Integer

The corresponding C code which calls it is

    void ddelap_f(double *x, int nx, double *a, int na, double *b, int
nb, double *l, int nl,
                  int *lg, double *ret);

and the actual code called by R is:

    extern SEXP ddelap_C(SEXP x, SEXP alpha, SEXP beta, SEXP lambda, SEXP lg){
      const int nx = LENGTH(x);
      const int na = LENGTH(alpha);
      const int nb = LENGTH(beta);
      const int nl = LENGTH(lambda);
      SEXP ret;
      PROTECT(ret = allocVector(REALSXP, nx));
      ddelap_f(REAL(x), nx, REAL(alpha), na, REAL(beta), nb,
REAL(lambda), nl, INTEGER(lg), REAL(ret));
      UNPROTECT(1);
      return(ret);
    }

Also, I don't use the "automatic" concatenation of the "c_" to my
Fortran functions, but name them individually and use Fortran 2003 ISO
C bindings (except for booleans).

If it helps, take a look at the source code at
<https://bitbucket.org/aadler/delaporte>. As of now, it passes all R
checks and so must have appeased the registration requirements 8-)

Thank you,

Avi
Rolf Turner
# 
Thanks for trying, but your example blows me away completely.

cheers,

Rolf
On 23/04/17 14:47, Avraham Adler wrote:
Duncan Murdoch
#
On 22/04/2017 5:25 PM, Rolf Turner wrote:
I think renaming the Fortran would make more sense:  the R routine may 
be used by your package users, whereas the Fortran routine may only be 
used by the R routine.  But there are probably lots of exceptions to this.
Not immediately, but a couple of potential ones:

  - If you change the type of something in the future, "void *" won't 
care, but the compiler might catch a the change if you were more explicit.

  - R CMD check generally becomes stricter over time, so it's possible 
it will complain about "void *" in the future.
Looks like

extern void F77_NAME(mnnd)(double *, double *, int *, double *, double *);

to me.

Duncan Murdoch
Rolf Turner
#
On 23/04/17 21:57, Duncan Murdoch wrote:
<SNIP>
Hmmm.  Don't follow this.  Sorry to be slow.  It just seemed easier to 
me to fiddle with the R code than with the Fortran code.  I think I 
could go either way.
Yeah, it was something like the latter that nagged the back of my mind. 
So I really would like to change the "void *" constructions.
Hence the following may be very helpful.
Looks quite plausible.  I'll give it a try and report back.  (Later; not 
tonight, Josephine.  Past my bedtime. :-) )  My recollection is that I 
tried something like that, but said "integer" rather than "int".
Which is, I guess, Fortran-ese rather than C-ese.  And I guess the 
latter is called for.  In retrospect, I was being silly, since I said 
"double" (C-ese) rather than "double precision" (Fortran-ese).  So 
saying "int" would have been consistent.  Psigh!

Thanks.

cheers,

Rolf
Duncan Murdoch
#
On 23/04/2017 6:38 AM, Rolf Turner wrote:
In one common case, the Fortran routine is written specifically as an 
interface to R, and is only called by the R routine that was written as 
that end of the interface.  That's what I was talking about.  But there 
are also situations where you have a Fortran subroutine that is used for 
many purposes besides the call from R; then renaming it would be more 
trouble.
Good luck!

Duncan Murdoch
Rolf Turner
#
On 23/04/17 23:05, Duncan Murdoch wrote:

            

      
      
      
    

        
One more (I hope it's the last!) question:

One of my subroutines has an argument of type *logical*.  There is no 
logical type in C.  So, since I am perforce using C-speak, I cannot 
change "void *" to "void logical".

I have a (very vague) understanding that in C one uses variables of int 
type (taking the values 0, for FALSE, and 1, for TRUE) as logical variables.

On that understanding I took a punt and replaced "void *" by "int *" for
the logical type variable.  The package built and passed

     "R CMD check --as-cran"

so it seems that this is OK.  Is this the Right Thing To Do?  Are there 
any (obvious?) lurking perils?

Thanks.

cheers,

Rolf

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 24 April 2017 at 10:18, Rolf Turner wrote:

        
| One more (I hope it's the last!) question:
| 
| One of my subroutines has an argument of type *logical*.  There is no 
| logical type in C.  So, since I am perforce using C-speak, I cannot 
| change "void *" to "void logical".
| 
| I have a (very vague) understanding that in C one uses variables of int 
| type (taking the values 0, for FALSE, and 1, for TRUE) as logical variables.
| 
| On that understanding I took a punt and replaced "void *" by "int *" for
| the logical type variable.  The package built and passed
| 
|      "R CMD check --as-cran"
| 
| so it seems that this is OK.  Is this the Right Thing To Do?  Are there 
| any (obvious?) lurking perils?

I think you are allowed to use C99 [1] which has it -- see eg

http://stackoverflow.com/questions/4159713/how-to-use-boolean-datatype-in-c

Dirk

[1] Section 1.6.4 opems with

    1.6.4 Portable C and C++ code
    -----------------------------

    Writing portable C and C++ code is mainly a matter of observing the
    standards (C99, C++98 or where declared C++11/14) and testing that
    extensions (such as POSIX functions) are supported.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Rolf Turner
    

    

      
      #
    

  


  

      
On 24/04/17 10:31, Dirk Eddelbuettel wrote:

        

            

      
      
      
    

        
Ah, but I'm *not* using C at all, I'm using Fortran.  So I think that 
raising the possibility of using C99 is a communist fish[1].  I just 
want to make sure that my (modified) init.c is syntactically correct and 
robust for implementing the registration of my *Fortran* routines.

cheers,

Rolf

[1] Red herring.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 24 April 2017 at 10:45, Rolf Turner wrote:

        
| On 24/04/17 10:31, Dirk Eddelbuettel wrote:

        
| > On 24 April 2017 at 10:18, Rolf Turner wrote:

        
| > | One more (I hope it's the last!) question:
| > |
| > | One of my subroutines has an argument of type *logical*.  There is no
| > | logical type in C.  So, since I am perforce using C-speak, I cannot
| > | change "void *" to "void logical".
| > |
| > | I have a (very vague) understanding that in C one uses variables of int
| > | type (taking the values 0, for FALSE, and 1, for TRUE) as logical variables.
| > |
| > | On that understanding I took a punt and replaced "void *" by "int *" for
| > | the logical type variable.  The package built and passed
| > |
| > |      "R CMD check --as-cran"
| > |
| > | so it seems that this is OK.  Is this the Right Thing To Do?  Are there
| > | any (obvious?) lurking perils?
| >
| > I think you are allowed to use C99 [1] which has it -- see eg
| >
| > http://stackoverflow.com/questions/4159713/how-to-use-boolean-datatype-in-c
| >
| > Dirk
| >
| > [1] Section 1.6.4 opems with
| >
| >     1.6.4 Portable C and C++ code
| >     -----------------------------
| >
| >     Writing portable C and C++ code is mainly a matter of observing the
| >     standards (C99, C++98 or where declared C++11/14) and testing that
| >     extensions (such as POSIX functions) are supported.
| 
| Ah, but I'm *not* using C at all, I'm using Fortran.  So I think that 
| raising the possibility of using C99 is a communist fish[1].  I just 
| want to make sure that my (modified) init.c is syntactically correct and 

I would be surpised if init.c was Fortran. Anyway...

| robust for implementing the registration of my *Fortran* routines.

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Duncan Murdoch
    

    

      
      #
    

  


  

      
On 23/04/2017 6:18 PM, Rolf Turner wrote:

            

      
      
      
    

        
For questions like this, see Writing R Extensions section 5.2. 
"Logical" in R is "int *" in C, and "INTEGER" in Fortran.

This is important:  *there is no way to pass a Fortran "LOGICAL" from R 
to Fortran*.

The issue is that different Fortran compilers store LOGICAL in different 
ways.  Some are equivalent to int with 0 for FALSE, all else for TRUE, 
but not all are.  So don't rely on it.

If you need to call a Fortran subroutine that takes a LOGICAL argument, 
you need to write another Fortran subroutine that takes an INTEGER 
argument.  Then compute "arg NE 0" (or whatever the Fortran version is 
for testing not equal to zero) and pass that to the original routine.

            

      
      
      
    

        
It's really "nonzero for TRUE", hence the test above.

            

      
      
      
    

        
Lots of lurking perils.  You got lucky; don't count on it.

Duncan Murdoch

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Rolf Turner
    

    

      
      #
    

  


  

      
On 24/04/17 11:22, Dirk Eddelbuettel wrote:

        

            

      
      
      
    

        
It isn't of course.  But it is the device used for "registering 
routines" of all both flavours (i.e. both C and Fortran).

cheers,

Rolf

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Avraham Adler
    

    

      
      #
    

  


  

      
On Sun, Apr 23, 2017 at 7:36 PM, Duncan Murdoch

        
<murdoch.duncan at gmail.com> wrote:

            

      
      
      
    

        
[SNIP]

            

      
      
      
    

        
I went through this very headache over the last few months. Even using
Fortran 2003 ISO C bindings, I ended up having to convert to integers
for Solaris SPARC. I know you are using .Fortran() and not .Call(),
but perhaps you would find this commit of some value
<https://bitbucket.org/aadler/delaporte/commits/709ab65696e61d18585466ac700cf2856dedcb35>,

Good luck!

Avi

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 24 April 2017 at 11:47, Rolf Turner wrote:

        
|

        
| On 24/04/17 11:22, Dirk Eddelbuettel wrote:

        
| 
| > I would be surpised if init.c was Fortran. Anyway...
| 
| It isn't of course.  But it is the device used for "registering 
| routines" of all both flavours (i.e. both C and Fortran).

And per Duncan's last (and his earlier emails) maybe you need to call from R
into C (for finer control over the interface) and only then call your Fortran
worker function.  To hide it, and its plausibly more limited interface,
entirely from R.

But I really have no firm idea (and tend to have none of these issues as I
use C++ most of the time from R) so I get out of the way now.

Best of luck,  Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Rolf Turner
    

    

      
      #
    

  


  

      
On 24/04/17 11:36, Duncan Murdoch wrote:

            

      
      
      
    

        
Hoo-boy!!!  Some serious code-rewriting called for!

Thanks for setting me straight on this.

Is it possible that R CMD check could be set up to issue an error, or at 
least a warning, if it finds "logical" or "as.logical" in the argument 
lists of calls to .Fortran()?

Seems like it would be a Good Idea if it is indeed possible.

cheers,

Rolf

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Avraham Adler
    

    

      
      #
    

  


  

      
On Sun, Apr 23, 2017 at 7:51 PM, Dirk Eddelbuettel <edd at debian.org> wrote:

            

      
      
      
    

        
What I found useful was Drew Schmidt's example Romp. Where he also
recommends against .Fortran() in general due to performance overheads.

<https://github.com/wrathematics/Romp>

Avi

  
  


  


      

    

    

      
      

        


  

        

      Duncan Murdoch
    

    

      
      #
    

  


  

      
On 23/04/2017 7:53 PM, Rolf Turner wrote:

            

      
      
      
    

        
The problem is at the other end.  Passing an R logical is fine; it just 
turns into an INTEGER on its way to Fortran.

I doubt if it is feasible to detect the problem in Fortran, but static 
analysis keeps getting better, so I could be wrong now or in a while.

Duncan Murdoch

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Rolf Turner
    

    

      
      #
    

  


  

      
On 24/04/17 12:39, Duncan Murdoch wrote:

            

      
      
      
    

        
Yes, but surely this is an indication that the Fortran code being called 
"expects" a logical and thereby might screw up if its way of
converting integers to logicals were problematic.

            

      
      
      
    

        
If it is, or should become, feasible, it seems to me that it should be 
done.   OTOH I gather from other posts that the .Fortran() interface 
will eventually be removed, so the problem (in this form) will go away.

The *real* problem will then be to write C wrappers for all our Fortran 
routines that are (directly) called by R functions.  And this is a task 
that is impossible for people like me who go catatonic when they see C 
code.  So at that point I will just give up and go to the pub.

<SNIP>

cheers,

Rolf