Skip to content

‘:::’ call

15 messages · Marc Schwartz, Gavin Simpson, Duncan Murdoch +6 more

#
I have a package (TSdbi) which provides end user functions that I 
export, and several utilities for plugin packages (e.g. TSMySQL) that I 
do not export because I do not intend them to be exposed to end users. I 
call these from the plugin packages using TSdbi:::  but that now 
produces a note in the checks:

* checking dependencies in R code ... NOTE
Namespace imported from by a ?:::? call: ?TSdbi?
   See the note in ?`:::` about the use of this operator. :: should be
   used rather than ::: if the function is exported, and a package
   almost never needs to use ::: for its own functions.

Is there a preferred method to accomplish this in a way that does not 
produce a note?

Thanks,
Paul
#
On Aug 28, 2013, at 11:15 AM, Paul Gilbert <pgilbert902 at gmail.com> wrote:

            
Paul,

See this rather lengthy discussion that occurred within the past week:

  https://stat.ethz.ch/pipermail/r-devel/2013-August/067180.html

Regards,

Marc Schwartz
#
Paul, this was discussed at length only a couple of days ago. See this thread:

http://comments.gmane.org/gmane.comp.lang.r.devel/34100

If I follow you, I think a change has been made that doesn't NOTE if
the use of `:::` is to packages for which you are also the maintainer.
But read the thread as the change is mentioned in there somewhere.

HTH

G
On 28 August 2013 10:15, Paul Gilbert <pgilbert902 at gmail.com> wrote:

  
    
#
On 13-08-28 12:29 PM, Marc Schwartz wrote:
I did follow the recent discussion, but no one answered the question "Is 
there a preferred method to accomplish this?" (I suppose the answer is 
that there is no other way, given that no one actually suggested 
anything else.) Most of the on topic discussion in that thread was about 
how to subvert the CRAN checks, which is not what I am trying to do and 
was also pointed out as a bad idea by Duncan. The substantive response was

 >r63654 has fixed this particular issue, and R-devel will no longer
 >warn against the use of ::: on packages of the same maintainer.
 >
 >Regards,
 >Yihui

but that strikes me as a temporary work around rather than a real 
solution: suppose plugins are provided by a package from another maintainer.

Since CRAN notes have a habit of becoming warnings and then errors, it 
seems useful to identify the preferred legitimate approach while this is 
still a note. That would save work for both package developers and CRAN 
maintainers.

My thinking is that there is a need for a NAMESPACE directive something 
like limitedExport() that allows ::: for identified functions without 
provoking a CRAN complaint when packages use those functions. But there 
may already be a better way I don't know about. Or perhaps the solution 
is to split the end user functions and the utilities for plugin packages 
into two separate packages?

Paul
#
On 28/08/2013 2:50 PM, Paul Gilbert wrote:
I don't see a need for that.  The reason ::: is bad is because 
non-exported functions may change without notice, breaking the package 
that uses them, and possibly giving the users of that package bad 
results.  If you're the maintainer of both the donor and user of the 
non-exported function, then you can be expected to keep them in sync, 
hence r63654.  If those are different people, then the donor had better 
indicate that the function is safe to use.  That's what export() does.

If you are worried about a cluttered listing of functions and help 
pages, you can use an initial "." in the name of the object, or use 
\keyword{internal} in the .Rd file.  I forget the full list of effects 
of each of these, but using both should make your function nearly 
invisible, while still exported if it is listed as such in the NAMESPACE 
file.

Duncan Murdoch
#
On 28/08/2013 3:06 PM, Kasper Daniel Hansen wrote:
And as a potential user of your package, I don't want you to use ::: if 
you don't have control over the other package, because its author might 
unwittingly change it in a way that causes me to get incorrect results.

If you want to use :::, go ahead, just don't put your package on CRAN, 
where I get most of my packages.  Then we'll both be happy.

Duncan Murdoch
#
If this issue is going to be solved at all, it might end up as yet
another "hack" like utils::globalVariables just to "fix" R CMD check
which was trying to fix things that were not necessarily broken.

To be clear, I was not suggesting subvert this check. What I was
hoping is a way to tell CRAN that "Yes, I have read the documentation;
I understand the risk, and I want to take it like a moth flying into
the flames".

Many people have been talking about this "risk", and how about some
evidence? Who was bitten by :::? How many real cases in which a
package was broken by :::?

Yes, unexported functions may change, so are exported functions (they
may change API, be deprecated, add new arguments, change defaults, and
so on). Almost everything in a package is constantly evolving, and I
believe the correct way (and the only way) to stop things from being
broken is to write enough test cases. When something is broken, we
will be able to know that. Yes, we may not have control over other
people's packages, but we always have control over our own test cases.
IMHO, testing is the justification of CRAN's reputation and quality,
and that is a part of what CRAN does.

In God we trust, and everyone else should bring tests.

Regards,
Yihui
--
Yihui Xie <xieyihui at gmail.com>
Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA
On Wed, Aug 28, 2013 at 1:50 PM, Paul Gilbert <pgilbert902 at gmail.com> wrote:
#
I may have confused things by referring to ':::' which everyone reads as 
not exported, not documented, not part of the API, constantly changing, ...

In my mind, the real question is about two levels of exporting, one to 
other package developers, and another to end users. In both cases they 
are part of the "API", relatively constant, and documented. (I try to 
document even internal functions, otherwise I can't remember what they do.)

So far, I see three possible solutions:

   1/ R adds another namespace directive allowing certain functions to 
be "exported" differently, possibly just by causing the checks to be 
silent about ::: when those functions are used in that way by other 
packages.

  2/ The package gets split in two, one for use by other packages and 
one for use by end users.

  3/ Some functions are exported normally but hidden by using "." in the 
beginning of their names. Other package maintainers would know they 
exist, but end users would not so easily find them. (Duncan's other 
suggestion of using \keyword{internal} in the .Rd file strikes me as 
problematic. I'm surprised CRAN checks do not already object to 
functions exported and documented with \keyword{internal}.)

Paul
On 13-08-28 03:44 PM, Yihui Xie wrote:
#
Why? I think this is exactly the use case of \keyword{internal}.
#
<< Extracted from email trail below >>
Fine. BTW, when will CRAN be removing the C(omprehensive) from its name?

CRAN can never offer any guarantee that code works properly. Lots of code on CRAN doesn't, that's why packages have bug-fix releases all the time. And packages sometimes change the functionality even of functions that they *do* export. I certainly don't blindly trust stuff I download from CRAN; code aside, lots of it has excruciating documentation which hardly inspires faith. Nor do I blame CRAN for hosting bad code. CRAN is just a library; libraries are not responsible for the content of the books they hold.

With respect specifically to ::: : to guard against functionality changes of the type mentioned, B just needs to use a strict version check on A's package in the NAMESPACE. When A updates on CRAN, B will realize that the new version needs to be checked (because B's package will refuse to load on systems using the new version of A), and B can check functionality and amend their version check.

Adding yet more options about limited exports etc is not a good idea, I think. R's structure is REALLY complicated, particularly when it comes to packages. There is already too much stuff, too many different ways to do things. That is probably one reason why there is so much confusion and indeed so many bad choices from maintainers. Adding yet more techno-fixes is liable to be counterproductive.

Still in the case of :::, I agree with Yihui Xie that CRAN should not bother worrying about it. A Note would be OK if it weren't that Notes have a nasty habit of somehow turning into Warnings. In that respect: why not just have a separate package that has all sort of checks that anyone might ever want (I find most of them useless, but I know others like them), but that is independent of CRAN checks?

More generally, while the principles that CRAN espouses seemed OK to me last time I checked (eg  packages shouldn't mess each other up), the same is not true for the approach CRAN takes to them. The plethora of checks makes the whole thing legalistic: the criterion becomes not "is this package any good?" but "does it pass checks AA-ZZ?" So, for a maintainer with very limited time for arguments and who knows their own code and is well aware that the CRAN checks produce lots of False Positives and also False Negatives (there is lots of bad code out there...), it's hardly surprising if there is every temptation to just pop in a workaround.

If there was a much tighter, and well-explained, set of checks, then I reckon there would be fewer workarounds overall (because of a cultural shift, not just in terms of superfluous checks that have been removed) and there wouldn't be the need for lengthy dialogs between maintainers and CRAN. Notwithstanding the enormous efforts that the CRAN people put in (whoever they may be...): CRAN might be better off working with human nature(s), not against it.

bye
Mark
#
On 13-08-28 05:13 PM, Hadley Wickham wrote:
From Writing R extensions "The special keyword ?internal? marks a page 
of internal objects that are not part of the package?s API" which 
suggests to me that a function with \keyword{internal} should not be 
exported, since that makes it part of the API. And, if it is really for 
internal use in a package, why would you export it? I think you are 
interpreting "internal" to mean internal to a group of packages, not 
internal to a package. But that is just the complement of what I am 
saying: there may be a need for two levels of export.

(Also, if you export it then you should document it, but for many 
maintainers \keyword{internal} is shorthand for I don't need to document 
this properly because no one is suppose to use it outside the package.)

Paul
#
On 13-08-28 9:00 PM, Kasper Daniel Hansen wrote:
How does this argument not apply to *all* tests done by CRAN? Should it 
stop doing any tests at all, because users can do them if they care?

Duncan Murdoch