Skip to content

[Rcpp-devel] macros RCPP_FUNCTION_...,

19 messages · Dirk Eddelbuettel, Darren Cook, Smith, Dale (Norcross) +2 more

Original
Romain Francois
# 
Hello,

Who is using the macros RCPP_FUNCTION_0, ..., RCPP_FUNCTION_65, ...
(i.e. all the macros that live in the preprocessor_generated.h file).

Their functionality is superseded by a much better way of doing it 
(modules).

I'd like to drop them at some point.

Obviously it is not happening overnight, because for starters our own 
RProtoBuf uses them. But I will update RProtoBuf so that it does not.
There is also uses in the RcppGSL vignette which I will update.



The file spans >6000 lines of generated code that we could do without. 
To put this into context, this represents about 10% of the lines of the 
.h files in Rcpp.

I don't think many people are using it, and I don't blame them. This was 
an interesting idea when we introduced it, but we have something better now.

Romain
Dirk Eddelbuettel
#
On 18 June 2013 at 16:26, Romain Francois wrote:
| Hello,
| 
| Who is using the macros RCPP_FUNCTION_0, ..., RCPP_FUNCTION_65, ...
| (i.e. all the macros that live in the preprocessor_generated.h file).
| 
| Their functionality is superseded by a much better way of doing it 
| (modules).
| 
| I'd like to drop them at some point.
| 
| Obviously it is not happening overnight, because for starters our own 
| RProtoBuf uses them. But I will update RProtoBuf so that it does not.
| There is also uses in the RcppGSL vignette which I will update.
| 
| 
| 
| The file spans >6000 lines of generated code that we could do without. 
| To put this into context, this represents about 10% of the lines of the 
| .h files in Rcpp.
| 
| I don't think many people are using it, and I don't blame them. This was 
| an interesting idea when we introduced it, but we have something better now.

Hm, I think my preference would be for a rather long period of deprecation --
at least a year if not more -- before declaring it defunct.

Even then, maybe we should consider to not remove them as we have a good
tradition of maintaining public APIs going forward. So my suggestion would be
to deactivate them behind an #if with a new configuration #define in
RcppConfig.

APIs are contracts we have with users.  I would prefer to see strong reasons
for chance, rather than a style preference.  

Now, 10% of code is of course a pretty good reason, but is it really making
that much of a difference?  The Rcpp package is large, and will remain large
either way.  An added #define should make compilaton faster which is good.

Other views?

Dirk
Romain Francois
# 
Le 18/06/13 19:50, Dirk Eddelbuettel a ?crit :
For what it is worth. I just downloaded and unpacked source code for 113 
packages using Rcpp, using a modified version of your runRcppDepends.

Then:

romain at naxos /tmp/stuff $ grep -r RCPP_FUNCTION *
RcppGSL/vignettes/RcppGSL/RcppGSL-intro.Rnw:% RCPP_FUNCTION_1( int, 
sum_gsl_vector_int, RcppGSL::vector<int> vec){
RcppGSL/vignettes/RcppGSL/RcppGSL-intro.Rnw:% RCPP_FUNCTION_1( double, 
gsl_vector_sum_2, Rcpp::List data ){
minqa/ChangeLog:	* src/minqa.cpp: Expand RCPP_FUNCTION_X macros to make 
debugging

So apart from:
- some vignette code that can easily be replaced by something using // 
[[Rcpp::export]] and be much easier to understand.
- some comment in  minqa

No one uses it.

Doing a long deprecating period over something nobody uses or care about 
is a good way to forget to do it.

OTOH, removing them deals with the issue right now. We might get -ve 
feedback from people who are in fact using it. In which case we can 
propose to either
- fix their code using something simpler, easier to understand, better
- let them include the 6600 lines of clutter

Again, this code was a good idea when we made it, not anymore.
This is more clutter, we have to document this defined.
This was just me stumbling upon the code and going: we don't need this 
stuff anymore, let's get it out of the way.

If we keep it we surely need to document it as part of this contract you 
mention.
This is an opportunity for a quick diet. Just before the summer.

Of course it will stay big, see
http://gallery.r-enthusiasts.com/graph/Evolution_of_Rcpp_code_size_165

Anyway, too much time wasted already writing this email, but I remain 
confident we can do something about that. We even managed to get the 
classic api out of the way after all.
Dirk Eddelbuettel
#
On 18 June 2013 at 21:05, Romain Francois wrote:
| For what it is worth. I just downloaded and unpacked source code for 113 
| packages using Rcpp, using a modified version of your runRcppDepends.

Our users comprise a much larger set than those visible at CRAN. As such not
a "proof", merely an operational test.
 
You know full well that we have users who are eg active on this list yet who
do not drop packages onto CRAN.

| No one uses it.

That is unclear. 

all_use_of(Rcpp) >> cran_use_of(Rcpp)
 
Dirk
Romain Francois
# 
Le 18/06/13 21:12, Dirk Eddelbuettel a ?crit :
sure.
Darren Cook
# 
Looking good in a bikini is always important. :-)

That no-one uses it in any CRAN package is also compelling. (I just
searched on StackOverflow and no mention there either.)
Dirk wrote:

        
Romain wrote:

            

      
      
      
    

        
I think this is the most compelling reason. It seems almost undocumented.

https://www.google.co.jp/search?q="RCPP_FUNCTION"
https://www.google.co.jp/search?q="RCPP_FUNCTION_0"

There is something here:
 http://dirk.eddelbuettel.com/blog/2010/05/17/

(BTW, shouldn't "RCPP_FUNCTION_VOID_2" have been "RCPP_FUNCTION_VOID_1" ?)

Can those examples be mechanically converted to use Rcpp modules,
Romain? If so, documenting how to convert might be sufficient, and
anyone who gets hit by the removal can spend 20 minutes updating their
code to work.

Darren

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 19 June 2013 at 09:33, Darren Cook wrote:

        
| > This is an opportunity for a quick diet. Just before the summer.
| 
| Looking good in a bikini is always important. :-)
| 
| That no-one uses it in any CRAN package is also compelling. (I just
| searched on StackOverflow and no mention there either.)
|

        
| Dirk wrote:

        
| >> APIs are contracts we have with users.  I would prefer to see strong
| >> reasons for change, rather than a style preference.
|

        
| Romain wrote:

        
| > If we keep it we surely need to document it as part of this contract
| 
| I think this is the most compelling reason. It seems almost undocumented.

For Pete's sake if that rule held half of Rcpp would be gone tomorrow.

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Romain Francois
    

    

      
      #
    

  


  

      
Le 19/06/13 02:33, Darren Cook a ?crit :

            

      
      
      
    

        
glad you enjoyed the visual

            

      
      
      
    

        
yes. so the single very well hidden bit of documentation we have about 
this is wrong.

            

      
      
      
    

        
I'm not writing mechanics for something that is never going to be used.

What do you want to see:

- this:

RCPP_FUNCTION_2( int, foobar, int x, int y){
	    return x + y ;
	  }

or this:

// [[Rcpp::export]]
int foobar( int x, int y){
     return x + y ;
}

The attributes feature is awesome. Anyone who reads c++ can understand 
what goes on. Not the same story with the macro.

I know their value, I wrote the code. I know what they are for. But we 
don't need them anymore.

            

      
      
      
    

        
Yes. And I'm happy to so it for every single one of them. Of course, I'm 
not taking a big risk because nobody uses this.

If we leave the code as it is, the hypothetical user that does not exist 
is happy.

If we remove it, the same hypothetical user is first going to get upset, 
and then he will be happy to have better code when I have fixed it.

For everyone else, they are going to have an Rcpp with less clutter and 
a better ratio of [stuff that is in the package] / [stuff that is 
documented].


This in my view is positive. Of course eventually I don't care all that 
much. This was just a random idea.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Smith, Dale (Norcross)
    

    

      
      #
    

  


  

      
Hi all,

I've been reading this with some interest. I suggest:

* Announce an api change via Stack Overflow, blog, list, etc and put the announcement into the NEWS for the next release.
* Announce an effective date on the change and outline the reasons.

I think this should satisfy everyone and give users fair warning that a change is coming that may break their code.

Dale Smith, Ph.D.
Senior Financial Quantitative Analyst
Risk & Compliance
Fiserv
Office: 678-375-5315
www.fiserv.com

-----Original Message-----
From: rcpp-devel-bounces at r-forge.wu-wien.ac.at [mailto:rcpp-devel-bounces at r-forge.wu-wien.ac.at] On Behalf Of Romain Francois
Sent: Tuesday, June 18, 2013 9:37 PM
To: rcpp-devel at r-forge.wu-wien.ac.at
Subject: Re: [Rcpp-devel] macros RCPP_FUNCTION_...,

Le 19/06/13 02:33, Darren Cook a ?crit :

            

      
      
      
    

        
glad you enjoyed the visual

            

      
      
      
    

        
yes. so the single very well hidden bit of documentation we have about this is wrong.

            

      
      
      
    

        
I'm not writing mechanics for something that is never going to be used.

What do you want to see:

- this:

RCPP_FUNCTION_2( int, foobar, int x, int y){
	    return x + y ;
	  }

or this:

// [[Rcpp::export]]
int foobar( int x, int y){
     return x + y ;
}

The attributes feature is awesome. Anyone who reads c++ can understand what goes on. Not the same story with the macro.

I know their value, I wrote the code. I know what they are for. But we don't need them anymore.

            

      
      
      
    

        
Yes. And I'm happy to so it for every single one of them. Of course, I'm not taking a big risk because nobody uses this.

If we leave the code as it is, the hypothetical user that does not exist is happy.

If we remove it, the same hypothetical user is first going to get upset, and then he will be happy to have better code when I have fixed it.

For everyone else, they are going to have an Rcpp with less clutter and a better ratio of [stuff that is in the package] / [stuff that is documented].


This in my view is positive. Of course eventually I don't care all that much. This was just a random idea.

--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30

R Graph Gallery: http://gallery.r-enthusiasts.com

blog:            http://blog.r-enthusiasts.com
|- http://bit.ly/Zs97qg  : highlight 0.4.1
`- http://bit.ly/10X94UM : Mobile version of the graph gallery

_______________________________________________
Rcpp-devel mailing list
Rcpp-devel at lists.r-forge.r-project.org
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

  
  


  


      

    

    

      
      

        


  

        

      Romain Francois
    

    

      
      #
    

  


  

      
Sure. That is essentially what Dirk proposed. That is fine.

What I've done is add this section in our NEWS file:

     \item Deprecation plans:
     \itemize{
       \item The set of macros \code{RCPP_FUNCTION_} etc ... from the
       \code{preprocessor_generated.h} file will be deprecated in the 
next version
       of \code{Rcpp}, i.e they willl still be available but will 
generate some
       warning in addition to their expected behavior.
       In release +2 the macros will be removed
       from \code{Rcpp}. Users of these macros (if any) should start 
replacing them
       with more up to date code, such as using attributes or modules.
     }


So:
- when we release Rcpp 0.10.4 nothing changes. people can still enjoy 
using this feature, if anyone is.
- when we release Rcpp 0.10.5 the feature is still there, but with extra 
annoying warnings
- when we release Rcpp 0.10.6 the feature is gone.

Considering we release now approximately quarterly, this leaves a period 
between 6 and 9 months before the feature is gone.

I hope everybody will be ok with that. This is more work, and I actually 
need to remember to do it, but that is a good compromise.

Romain


Le 19/06/13 14:35, Smith, Dale (Norcross) a ?crit :

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 19 June 2013 at 15:12, Romain Francois wrote:

        
| So:
| - when we release Rcpp 0.10.4 nothing changes. people can still enjoy 
| using this feature, if anyone is.
| - when we release Rcpp 0.10.5 the feature is still there, but with extra 
| annoying warnings
| - when we release Rcpp 0.10.6 the feature is gone.
| 
| Considering we release now approximately quarterly, this leaves a period 
| between 6 and 9 months before the feature is gone.

The period between a (released) announcement and the removal will not be less
than 12 months.

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Romain Francois
    

    

      
      #
    

  


  

      
Le 19/06/13 15:41, Dirk Eddelbuettel a ?crit :

            

      
      
      
    

        
ok, fine. whatever works.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Hadley Wickham
    

    

      
      #
    

  


  

      
On Wed, Jun 19, 2013 at 8:41 AM, Dirk Eddelbuettel <edd at debian.org> wrote:

            

      
      
      
    

        
For API breaking changes, I think it's good practice to increment a
bigger version number - so I think you should remove it until 0.11.0.

It's also not like it's impossible to get old versions from Rcpp for
CRAN, so if someone's code does break they do already have recourse.

Hadley


--
Chief Scientist, RStudio
http://had.co.nz/

  
  


  


      

    

    

      
      

        


  

        

      Romain Francois
    

    

      
      #
    

  


  

      
Le 19/06/13 15:49, Romain Francois a ?crit :

            

      
      
      
    

        
I've updated the wording to this:

       \item The set of macros \code{RCPP_FUNCTION_} etc ... from the
       \code{preprocessor_generated.h} file will be deprecated in the 
next version
       of \code{Rcpp}, i.e they will still be available but will 
generate some
       warning in addition to their expected behavior.

       In the first release 12 months after now, the macros will be removed
       from \code{Rcpp}.

       Users of these macros (if any) should start replacing them
       with more up to date code, such as using attributes or modules.
     }



Feel free to change it if this does not suit you.

This just makes it slightly more difficult to do, because with my 
proposed approach, I could just have made the changes right after each 
release. But I'll comply with those requirements.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Romain Francois
    

    

      
      #
    

  


  

      
Le 19/06/13 15:50, Hadley Wickham a ?crit :

            

      
      
      
    

        
We were waiting for the book release to bump up to 1.0.0 right ?
Rcpp is mature enough now to deserve a 1.0.0.

            

      
      
      
    

        
Definitely. And if they want to host 6600 lines of auto generated code 
in their package, they can just pick it up from an older version.

Anyway. I'm fine with the compromise of letting these go with the first 
release that comes after a year after 0.10.4 is released. Just feels 
like a lot of bureaucracy for not much.

I think this process would be perfect if we were to do something radical 
with the api. For example, factor out sugar in its own package (I'm not 
saying it is planned, but this is a self contained entity that could 
very well have an existence of its own and benefit from shorter release 
cycles, etc ...).

But for something like this, removing code that is very unlikely used, I 
would have liked us to be more reactive. And I want a pony.

Overall it is good to have balance in a team. So no issue.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 19 June 2013 at 08:50, Hadley Wickham wrote:

        
| It's also not like it's impossible to get old versions from Rcpp for
| CRAN, so if someone's code does break they do already have recourse.

Correct. That is the ultimate defense with Open Source.

Breaking an API is extremely poor style and annoying as hell to everybody.
We all agree on that -- and what Romain proposed (ie removing unused cruft)
is generally a good rather than thing. But one has to be extra careful to not
accidentally create the former too.

The burden only grows with the number of users, and some users do have long
and slow upgrade cycles.  Ie we just learned about a commercial entities
being forced to use R 2.12.0. They are of course safe from all that because
they been excluded from any modern Rcpp for years already :)

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 19 June 2013 at 16:07, Romain Francois wrote:

        
| We were waiting for the book release to bump up to 1.0.0 right ?

Not really.

| Rcpp is mature enough now to deserve a 1.0.0.

Maybe. Maybe not.  I think I learn towards 'not'.

| But for something like this, removing code that is very unlikely used, I 
| would have liked us to be more reactive. And I want a pony.

Did you mean 'proactive' here?   And yes, we all want a pony.

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Romain Francois
    

    

      
      #
    

  


  

      
Le 19/06/13 16:14, Dirk Eddelbuettel a ?crit :

            

      
      
      
    

        
I'm more than happy to set up any support contract with those commercial 
entities if they want a version of Rcpp that is compatible with antique R.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Romain Francois
    

    

      
      #
    

  


  

      
Le 19/06/13 16:17, Dirk Eddelbuettel a ?crit :

            

      
      
      
    

        
Ah ok. I don't really care anyway. That would look nice though.

            

      
      
      
    

        
Well. Did you mean 'lean' here ?