An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20130822/a10129e8/attachment.pl>
legitimate use of :::
34 messages · Michael Friendly, Peter Dalgaard, Hadley Wickham +12 more
Messages 26–34 of 34
Maybe it is not a good idea for R CMD check to check ::: at all, and a warning in R-exts and ?':::' may well be enough. On the other hand, it is just so easy to get around :::, because everybody can see its source code:
`:::`
function (pkg, name)
{
pkg <- as.character(substitute(pkg))
name <- as.character(substitute(name))
get(name, envir = asNamespace(pkg), inherits = FALSE)
}
Then the package authors who really want to take the risk may start
another "hide and seek" game, e.g.
`%:::%` = function(pkg, fun) get(fun, envir = asNamespace(pkg),
inherits = FALSE)
'stats' %:::% 'Pillai'
Non-exported functions do not necessarily imply instability. Maybe it
is just because the author is too lazy to document them, or he/she did
not realize these functions happen to be useful for another author.
Although R-devel does not warn against ::: on the packages of the same
maintainer now, these maintainers may change in the future, or one
maintainer can be an author but not maintainer of another package,
from which he/she imports an unexported function, or package authors
have coordinated well with each other about the unexported functions.
I believe there are other legitimate reasons for :::, which might make
it difficult for R to cover all these cases, and also bring additional
communications between package authors and CRAN.
In conclusion, R CMD check cannot really stop :::, and ::: can be
there for good reasons, so how about just let it go?
Regards,
Yihui
--
Yihui Xie <xieyihui at gmail.com>
Web: http://yihui.name
Department of Statistics, Iowa State University
102 Snedecor Hall, Ames, IA
On Thu, Aug 22, 2013 at 9:02 PM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:
Hey guys, Because I was curious and had nothing else that I should have been doing (that second part is a lie), I looked into the namespace code. I have a working patch that implements importHiddenFrom. It doesn't currently check whether you then export that symbol (which should not be allowed) but that would be easy to implement. Is there any desire from R-core to have add the importHiddenFrom functionality? If so I can add the export check and submit the patch either here or on bugzilla. I figure its a long shot but hey, at least I now know how the namespace stuff works. I do agree with Peter Meilstrup that poking around at the internals of someone else's code is often not a good idea, but as others have pointed out it is done in practice in some fairly high-profile packages, and if its going to happen it seems like it would be nice to indicate as much in the NAMESPACE file. ~G On Thu, Aug 22, 2013 at 5:41 PM, Gray <gray at clhn.co> wrote:
Peter Meilstrup: (05:01PM on Thu, Aug 22) One most often encounters namespace conflicts at the user level, when
loading two packages that have no logical connection other than both bearing on your problem of the moment.
Unless I'm mistaken, you can reassign the hidden functions, ie fna <- joespackage:::usefulfunction fnb <- janespackage:::usefulfunction which is a little bit of a pain, but makes the user's code unambiguous. This also works with two colons for explicitly exported functions. -- Gray Calhoun, Assistant Professor of Economics at Iowa State http://gray.clhn.co (web)
To avoid the NOTEs (which often triggers a 'pls fix' upon submission to CRAN), I simply copied/pasted these functions to my package, but this seems wasteful.
An issue is how one acknowledges the author of the cut and pasted code. Assume that for one reason or another the original function can't easily be made available via NAMESPACEs and so to avoid CRAN submission complaints you are forced into cut and copying. If the copied code is somewhat substantial a short acknowledgment on a man page is insufficient. As far as I know the only alternative to list the author of the copied code in the Author field of your DESCRIPTION file. This makes the Author field ungainly (especially if the copied-from package has itself a lengthy Author field) and gives disproportionate credit to the author of the copied code over those in Depends. It would be good if there was a better mechanism in the DESCRIPTION file for acknowledging authors you copied code from. Tricky, lots of grey areas. On balance I think I'm in the camp of those who say ::: should allowed.
On 13-08-22 11:54 PM, Yihui Xie wrote:
Maybe it is not a good idea for R CMD check to check ::: at all, and a warning in R-exts and ?':::' may well be enough. On the other hand, it is just so easy to get around :::, because everybody can see its source code:
It's a really bad idea to write tricky code to subvert the tests. If the tests are wrong, you can argue for changes (and in this case you did, and changes were made), but if you can't give a convincing argument, you should follow the good practices that the repository policies enforce. Duncan Murdoch
`:::`
function (pkg, name)
{
pkg <- as.character(substitute(pkg))
name <- as.character(substitute(name))
get(name, envir = asNamespace(pkg), inherits = FALSE)
}
Then the package authors who really want to take the risk may start
another "hide and seek" game, e.g.
`%:::%` = function(pkg, fun) get(fun, envir = asNamespace(pkg),
inherits = FALSE)
'stats' %:::% 'Pillai'
Non-exported functions do not necessarily imply instability. Maybe it
is just because the author is too lazy to document them, or he/she did
not realize these functions happen to be useful for another author.
Although R-devel does not warn against ::: on the packages of the same
maintainer now, these maintainers may change in the future, or one
maintainer can be an author but not maintainer of another package,
from which he/she imports an unexported function, or package authors
have coordinated well with each other about the unexported functions.
I believe there are other legitimate reasons for :::, which might make
it difficult for R to cover all these cases, and also bring additional
communications between package authors and CRAN.
In conclusion, R CMD check cannot really stop :::, and ::: can be
there for good reasons, so how about just let it go?
Regards,
Yihui
--
Yihui Xie <xieyihui at gmail.com>
Web: http://yihui.name
Department of Statistics, Iowa State University
102 Snedecor Hall, Ames, IA
On Thu, Aug 22, 2013 at 9:02 PM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:
Hey guys, Because I was curious and had nothing else that I should have been doing (that second part is a lie), I looked into the namespace code. I have a working patch that implements importHiddenFrom. It doesn't currently check whether you then export that symbol (which should not be allowed) but that would be easy to implement. Is there any desire from R-core to have add the importHiddenFrom functionality? If so I can add the export check and submit the patch either here or on bugzilla. I figure its a long shot but hey, at least I now know how the namespace stuff works. I do agree with Peter Meilstrup that poking around at the internals of someone else's code is often not a good idea, but as others have pointed out it is done in practice in some fairly high-profile packages, and if its going to happen it seems like it would be nice to indicate as much in the NAMESPACE file. ~G On Thu, Aug 22, 2013 at 5:41 PM, Gray <gray at clhn.co> wrote:
Peter Meilstrup: (05:01PM on Thu, Aug 22) One most often encounters namespace conflicts at the user level, when
loading two packages that have no logical connection other than both bearing on your problem of the moment.
Unless I'm mistaken, you can reassign the hidden functions, ie fna <- joespackage:::usefulfunction fnb <- janespackage:::usefulfunction which is a little bit of a pain, but makes the user's code unambiguous. This also works with two colons for explicitly exported functions. -- Gray Calhoun, Assistant Professor of Economics at Iowa State http://gray.clhn.co (web)
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
1 day later
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20130824/d51fe8c2/attachment.pl>
1 day later
On 13-08-24 12:46 PM, Kasper Daniel Hansen wrote:
On Thu, Aug 22, 2013 at 8:27 PM, Peter Meilstrup Using ::: on a package you don't control can be more dangerous. For a
package author to choose to export a function to the public interface represents at least some assurance that that interface will be stable or slow-moving. But there are no implied assurances about how code in the private namespace might change behind the scenes. I might completely refactor the code and change the behavior of any private function between 0.0.1 releases, but I would not do that for exported functions.
This is true (that it could be dangerous), but sometimes, as a package author, I am willing to take this risk. Personally, I don't do this lightly, but sometimes there is no way around it, particular if the hidden function does some magic in its own NAMESPACE. It is not all functions that one can just easily copy over into you own package. It is fine to say that the use of ::: should be discouraged, it is another thing if it prevents you from submitting to CRAN (which I don't know for sure; I thought that Notes were ok?).
CRAN is a good place to send your packages because it does some quality control on the packages it accepts. As I said in my message to Yihui, if you think the test is wrong, you can argue about that and it might be changed, but if you can't convince the CRAN maintainers of the validity of your point, you shouldn't send your package there. There are plenty of other places (github, R-forge, etc.) where you can distribute it. Duncan Murdoch
I know it is really bad, but the so-called good approach can be more expensive than that, primarily because a package with a NOTE in R CMD check is likely to be rejected by CRAN, or authors have to justify the NOTE in the email. For this particular case, I can imagine at least one case which can be endless trouble to CRAN: if the package author A has talked to B to use a hidden function in B's package, and B thinks that function is relatively stable but does not want to export it; A may go ahead and use it via :::. Both A and B understand that unexported function well, then what should A do when submitting the package to CRAN? Should CRAN continue adding rules for such exceptions? In other words, "bad" practices almost always have exceptions and edge cases. Of course, the decision is always in CRAN's hands, and I will try to respect and follow it as a package author. Regards, Yihui -- Yihui Xie <xieyihui at gmail.com> Web: http://yihui.name Department of Statistics, Iowa State University 102 Snedecor Hall, Ames, IA On Fri, Aug 23, 2013 at 11:05 AM, Duncan Murdoch
<murdoch.duncan at gmail.com> wrote:
On 13-08-22 11:54 PM, Yihui Xie wrote:
Maybe it is not a good idea for R CMD check to check ::: at all, and a warning in R-exts and ?':::' may well be enough. On the other hand, it is just so easy to get around :::, because everybody can see its source code:
It's a really bad idea to write tricky code to subvert the tests. If the tests are wrong, you can argue for changes (and in this case you did, and changes were made), but if you can't give a convincing argument, you should follow the good practices that the repository policies enforce. Duncan Murdoch
`:::`
function (pkg, name)
{
pkg <- as.character(substitute(pkg))
name <- as.character(substitute(name))
get(name, envir = asNamespace(pkg), inherits = FALSE)
}
Then the package authors who really want to take the risk may start
another "hide and seek" game, e.g.
`%:::%` = function(pkg, fun) get(fun, envir = asNamespace(pkg),
inherits = FALSE)
'stats' %:::% 'Pillai'
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20130825/816465f7/attachment.pl>
On Sat, Aug 24, 2013 at 9:46 AM, Kasper Daniel Hansen
<kasperdanielhansen at gmail.com> wrote:
On Thu, Aug 22, 2013 at 8:27 PM, Peter Meilstrup Using ::: on a package you don't control can be more dangerous. For a
package author to choose to export a function to the public interface represents at least some assurance that that interface will be stable or slow-moving. But there are no implied assurances about how code in the private namespace might change behind the scenes. I might completely refactor the code and change the behavior of any private function between 0.0.1 releases, but I would not do that for exported functions.
This is true (that it could be dangerous), but sometimes, as a package author, I am willing to take this risk. Personally, I don't do this lightly, but sometimes there is no way around it, particular if the hidden function does some magic in its own NAMESPACE. It is not all functions that one can just easily copy over into you own package. It is fine to say that the use of ::: should be discouraged, it is another thing if it prevents you from submitting to CRAN (which I don't know for sure; I thought that Notes were ok?).
Closely related and particularly on which NOTEs are CRAN show stoppers and which are not: I find 'R CMD check' extremely useful and I believe it has greatly improved the quality of R and contributed packages including mine. The more checks/tests the better. No doubt about that. Keeping in mind the famous statement that "R is not CRAN" (or is it "CRAN is not R"), and knowing there is great overlap between R core and CRAN team members, it is still not easy to distinguish between what is R and what is CRAN and where they're heading. The CRAN Repository Policy [http://cran.r-project.org/web/packages/policies.html ; very useful addition] is very clear on 'R CMD check --as-cran' WARNINGs and ERRORs, but for NOTEs it leaves room for interpretation. For instance, it mentions that package with "significant notes" needs to be updated, but from experience I'd say it's up to the package developer to guess what CRAN means by "significant notes". It also gives sends the message that there is some leeway for the developer to publish a package with ("regular"?) NOTEs as long as s/he has considered their implications [cf. "If there are warnings or notes you cannot eliminate (for example because you believe them to be spurious) send an explanatory note as part of your covering email, or as a comment on the submission form."]. Unfortunately you won't find out until submission, which seems a waste of time for both the developer and CRAN. If CRAN accept certain 'R CMD check --as-cran' NOTEs but not others, may I propose some kind of mechanism to '--as-cran' that grep()/grep(invert=TRUE) which NOTEs are significant/non-significant for CRAN submissions/updates? That may be too much work to be exhaustive, but at least the ones that, to CRAN, are obviously rejected/accepted would be helpful. With 4700+ packages of CRAN with lots of daily submissions/updates, the CRAN team has a much better idea what's accepted or not, than each individual developer. If it is the case that CRAN consider the majority of the NOTEs to be "significant", I propose to clarify/be explicit about that in the CRAN Repository Policy document. ...and finally, thanks to the CRAN team [if they read this] for the work they put in daily. /Henrik PS. I am aware that proposals to CRAN should be sent to CRAN, but it's useful to have this discussion among R developers before going one-on-one with CRAN. If there is an agreement on the above, I'll send it off to the CRAN team.
Best,
Kasper
[[alternative HTML version deleted]]
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel