declaring package dependencies

On 13-09-14 12:19 PM, Paul Gilbert wrote:

On 13-09-14 09:04 AM, Duncan Murdoch wrote:

On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote:

On 13 September 2013 at 11:42, Paul Gilbert wrote:
| On 13-09-13 11:02 AM, Dirk Eddelbuettel wrote:
| > It's not so much Rcpp itself or my 20-ish packages but the fact
that we (as
| > in the Rcpp authors) now stand behind an API that also has to
accomodate
| > changes in R CMD check. Case in point is current (unannounced)
change that
| > makes all Depends: Rcpp become Imports: Rcpp because of the
NAMESPACE checks.
|
| I am a bit confused by this Dirk, so maybe I am missing something. I
| think this is still a "Note" in R-devel so you do have some time to
make
| the change, at least several months, maybe more. It is not quite
what I
| think of as an "announcement", more like a shot across the bow,
but it
| is also not "unannounced".

One package author [as in user of Rcpp and not an author of it] was
told by
CRAN this week to change his package and came to me for help -- so in
that
small way the CRAN "non-communication policy" is already creating more
work
for me, and makes me look silly as "I don't document what Rcpp-using
packages
need" as I sadly still lack the time machine or psychic powers to
infer what
may get changed this weekend.

| More importantly, I don't think that the requirement is
necessarily to
| change Depends: Rcpp to Imports: Rcpp, the requirement is to put
| imports(Rcpp) in the NAMESPACE file. I think this is so that the
package
| continues to work even if the user does something with the search
path.
| The decision to change Depends: Rcpp to Imports: Rcpp really
depends on
| whether the package author wants Rcpp functions to be available
directly

Rcpp is a bit of an odd-ball as you mostly need it at compile-time,
and you
require very few R-level functions (but there is package
initialization etc
pp).  We also only about two handful of functions, and those are for
functionality not all 135 packages use (eg Modules etc).

But the focus here should not be on my hobby package. The focus needs
to be
on how four CRAN maintainers (who do a boatload of amazing work
which is
_truly_ appreciated in its thoroughness and reach) could make the
life of
authors of 4800+ packages easier by communicating and planning a tad
more.

Let me paraphrase that:  "The CRAN maintainers do a lot of work, and it
helps me a lot, but if they only did a little bit more work it would
help me even more."

I suspect they'd be more receptive to suggestions that had them doing
less work, not more.

Actually, this is one of the parts that I do not understand. It seems to
me that it would be a lot less work for CRAN maintainers if the
implications and necessary changes to packages were explained a bit more
clearly in a forum like R-devel that many package developers actually
read regularly.

Then why don't you explain them?  They aren't secret.

I many not fully understand how much of the response to

package submission gets done automatically, but I do get the sense that
there is a fairly large amount of actual human time spent dealing with
just my submissions alone. If that is representative of all developers,
then CRAN maintainers don't have time to do much else. (The fact that
they do much more suggests I may not be representative.)

Two specific points have already been mentioned implicitly. CRAN
submission testing is often done at a higher/newer level using the
latest devel version. This results in lots of rejections for things that
I would fix before submission, if I knew about them.

Then why don't you test against R-devel before submitting?

  If the tests were

rolled out with R, and only later incorporated into CRAN submission
testing, I think there would be a lot less work for the CRAN
maintainers.

I am an R core member, not a member of CRAN.  If I make a change to R,
I'd like to know the effect it has on the corpus of CRAN and
Bioconductor packages, but I don't have the computing resources to run
it against all of those in a reasonable amount of time.  We have heard
from various corporate users of R (Revolution Computing and Google are
two) that they would like to contribute to the project, but we don't
actually see useful contributions from them.

I don't know if you personally have influence with anyone who has such
resources,

but surely some on this list do.  So why don't they write to
me and tell me how to use their computing resources to run proposed
changes against all published packages?  I think it's because it's
easier to do nothing than to do something.  It's certainly easier to say
that it's the CRAN maintainers' fault for poor communication than it is
to identify and address the problems.

Duncan Murdoch


  (This is ignoring the possibility that CRAN submission is

really the testing ground for the tests, and to prove the tests requires
a fair amount of manual involvement. I'm happy to continue contributing
to this -- I've often felt my many contribution is an endless supply of
bugs for the checkers to catch.)

The second point is that a facility like R-forge that runs the latest
checks, on many platforms, is really useful in order to reduce work for
both package developers and CRAN maintainers. With R-forge broken, the
implication for additional work for CRAN maintainers seems enormous. But
even with it working, not all packages are kept on R-forge, and with
package version dependencies R-forge does not really work. (i.e. I have
to get new versions of some packages onto CRAN before the new versions
of other packages will build on R-forge.)  Perhaps the package checking
part of R-forge should be separated into a pre-submission clearing house
to which packages are submitted. If they pass checks there then the
package developer could click on a submit button to do the actual
submission to CRAN. (Of course there needs to be a mechanism to plead
for the fact that the test systems do not have needed resources.)
Something like the daily, but with new pre-release versions of packages
might actually be better than the R-forge approach, for two reasons. One
is that package maintainers would only put packages there that they
think are actually working. (R-forge tries to build my svn copy even
when I know it is broken.) The second is that it would automatically
handle the version dependency problem, since package maintainers would
have the ability to put in place versions that should work together.
However, this does not need to be run daily. It only needs to be run
when the checks change, or for a package when the package changes.

Paul

Duncan Murdoch

| by users without them needing to specifically attach Rcpp. They are
| available with Depends but with Imports they are just used
internally in
| the package.
|
| So, one of us is confused. Usually it is me.

No, no, I usually keep you company.

Dirk