Skip to content

[R-pkg-devel] Two packages with the same generic function

20 messages · Bert Gunter, Neal Fultz, Tom Wainwright +6 more

#
Hi All,

Let's say there are two packages pkgA and pkgB, both of which have a generic function

foo <- function(x, ...)
   UseMethod("foo")

and pkgA has a method for objects of class "A":

foo.A <- function(x, ...)
   print(x)

and pkgB has a method for objects of class "B":

foo.B <- function(x, ...)
   plot(x)

Both packages export foo and their method and declare their respective S3 methods, so:

export(foo)
export(foo.A)
S3method(foo, A)

in NAMESPACE of pkgA and

export(foo)
export(foo.B)
S3method(foo, B)

in NAMESPACE of pkgB.

If a user loads pkgA first and then pkgB, this fails:

library(pkgA)
library(pkgB)
x <- 1:4
class(x) <- "A"
foo(x)

Error in UseMethod("foo") : 
  no applicable method for 'foo' applied to an object of class "A"

and vice-versa. Of course, pkgA::foo(x) works. Aside from pkgA importing foo() or vice-versa, is there some other clever way to make this work? In earlier versions of R (at least in 3.6.3), this used to work (i.e., the generic foo() from pkgB would find method foo.A() and vice-versa), but not since 4.0.0.

Best,
Wolfgang
#
On 22/06/2020 1:00 p.m., Viechtbauer, Wolfgang (SP) wrote:
Wouldn't the user have got a warning at this point about pkgB::foo 
masking pkgA::foo?
Can't one of the packages import the generic from the other package, and 
then declare the method as a method of that other generic?  That seems 
like the only thing that would make sense.  There's no reason to believe 
that pkgA::foo has anything whatsoever to do with pkgB::foo without this.

Duncan Murdoch
#
...
and just to add to the query, assume the author of pkg B did (does) not
know of pkg A and so, for example, could (did) not import any of pkg A's
content into B. Given that there are at the moment ~20,000 packages out
there, this does not seem to be an unreasonable assumption. One may even
further assume that the user may not know that (s)he has package B loaded,
as it may be a dependency of another package that (s)he uses. I certainly
don't keep track of all the dependencies of packages I use.

Under these assumptions, is there any more convenient alternative to
Wolfgang's pkgA:foo(x) explicit call under such assumptions? If pkgA has a
long name, what might one do?

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Mon, Jun 22, 2020 at 10:00 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

            

  
  
#
On 22/06/2020 1:40 p.m., Bert Gunter wrote:
It's always possible to make a new name, e.g.

fooA <- pkgA::foo
fooB <- pkgB::foo

If you are writing a package, this can be done in the NAMESPACE file, e.g.

importFrom(pkgA, fooA = foo)

though this doesn't appear to be documented in the usual places.

Duncan Murdoch
#
Another decent alternative is to submit the generic function to the
`generics` package - https://github.com/r-lib/generics - or create
your own API package.

Both package A and B can import the generic from it, and everything
works fine. It may also reduce your dependencies and/or build time.

This is much simpler than the workarounds we had tried to optionally
allow use of tidy() from DeclareDesign without transitively importing
the entire tidyverse, for example.

We had tried various ways of delegating from pkgA::foo.default to
pkgB::foo but it had some edge cases around the order that the user
attached the packages IIRC.

On Mon, Jun 22, 2020 at 11:01 AM Duncan Murdoch
<murdoch.duncan at gmail.com> wrote:
#
Yet another alternative is simply to prevent your second package from
overriding the previously defined generic. The basic problem is the ease
with which R allows overriding prior generic definitions (one of those bits
of bad behavior we in the USA used to call "a Bozo No-No"), which hides all
the previous methods, as demonstrated by the following code:
(Despite Murdoch's suggestion that overriding the generic SHOULD issue a
warning, it doesn't seem to in R 4.0.1.)

So, we might try protecting the generic definitions of "foo" in both
packages by enclosing them in something like:

tryCatch(invisible(methods("foo")), error = {foo <- function(x,...)
There's probably a more elegant way to accomplish this. This relies on
"methods" returning an error if "foo" has no defined methods, so it is not
redefined if their are previous methods. I haven't had time to try this in
the two-package example, but it might work, although I'm not sure how to
handle the Namespace declarations.

  Tom Wainwright
On Mon, Jun 22, 2020 at 10:41 AM Bert Gunter <bgunter.4567 at gmail.com> wrote:

            

  
  
#
On 22/06/2020 3:48 p.m., Tom Wainwright wrote:
Sure it does, if pkgA and pkgB both export the same name, then you get a 
warning when you attach the second one.  For example,

 > library(MASS)
 > library(dplyr)

Attaching package: ?dplyr?

The following object is masked from ?package:MASS?:

     select

The following objects are masked from ?package:stats?:

     filter, lag

The following objects are masked from ?package:base?:

     intersect, setdiff, setequal, union

Users don't get warned about overriding names in packages they've 
loaded, because that would just be irritating.

Duncan Murdoch
#
Hi Duncan: I maintain dynlm and your example is the exact reason I've been
getting emails from people regarding
it not working correctly. I've been telling them to load dplyr by using

library(dplyr, exclude = c("filter", "lag"))





On Mon, Jun 22, 2020 at 7:57 PM Duncan Murdoch <murdoch.duncan at gmail.com>
wrote:

  
  
#
"Users don't get warned about overriding names in packages they've
loaded, because that would just be irritating."

Is that also true if the package or generic is imported by another that
they load; or is a dependency of a package they load? If so, I would not
call it "just irritating" because if silent, how would they know?


Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Mon, Jun 22, 2020 at 5:58 PM Mark Leeds <markleeds2 at gmail.com> wrote:

            

  
  
#
On Tue, 23 Jun 2020, Bert Gunter wrote:

            
All Duncan is saying is that you don't get a notification if you do

     mean <- log

in the interpreter. If you attach a package that does this you would
get a notification (or an error if you configure your conflict
resolution options appropriately).

Best,

luke

  
    
#
OK. Thanks.

Bert Gunter
On Mon, Jun 22, 2020 at 7:51 PM <luke-tierney at uiowa.edu> wrote:

            

  
  
#
Thanks to all who replied and provided input.

@Duncan: Yes, of course there is the note about masking. In my experience, many people won't understand what this means, what the consequences are, and that pkgA::foo(x) and pkgB::foo(x) are workarounds. And yes, pkgA importing the foo() generic from pkgB or vice-versa is an obvious solution.

@Bert: Related to the suggestion that one package could import the generic from the other, you raise an important point. This in essence adds a whole package as a dependency for these two lines of code:

foo <- function(x, ...)
   UseMethod("foo")

And let's say pkgA imports this from pkgB and pkgB depends on 10 other packages? Does this mean that pkgA has now added 11 packages as dependencies for importing 2 lines of code? If so, that seems quite undesirable to me.

@Neal: A separate package with generic functions that pkgA and pkgB could import is an interesting suggestion, thanks!

@Tom: I might need some time to process what you are suggesting. What I am really hoping for (and not sure if your approach can handle this) is that the following works regardless of the order in which pkgA and pkgB are loaded:

x <- 1:4
class(x) <- "A"
foo(x)
class(x) <- "B"
foo(x)

Again, that used to be the case in R 3.6.3, but not anymore in 4.0.x.

Best,
Wolfgang
#
Am 23.06.20 um 10:00 schrieb Viechtbauer, Wolfgang (SP):
What makes this interesting is that there is no dependency on other 
packages in generics.

Remains the question which help page would be shown for help(foo).

Best, Guido
#
On 22/06/2020 10:17 p.m., Bert Gunter wrote:
I can't think of an example where this would be a problem.  If a package 
imports objects from another package, it doesn't affect the user's 
search list.  Maybe it affects what methods are available, but I can't 
see how it would change what generics are available.

Can you give an example of what you're worried about?

Duncan Murdoch
#
On 23/06/2020 4:22 a.m., Guido Schwarzer wrote:
If a package imports and then exports the generic, it would normally 
create a help page referring to the original one where the generic is 
defined.  So both pkgA and pkgB would probably both create help pages, 
and the help system would show a page listing them both plus the generic 
one, and asking the user to choose.

An example happens if you use

library(broom)
?tidy

The broom package links to a page that says "These objects are imported 
from other packages. Follow the links below to see their documentation."
One of the links is to the ?tidy page in the generics package.

You are allowed to say ?broom::tidy, and then you don't go to the list 
of possibilities, you go directly to the one you asked for.

Duncan Murdoch
#
Wait; I have options on how to configure conflict resolution? Can you 
tell me where to read more about this feature?

Thanks,
 ? Kevin
On 6/22/2020 10:51 PM, luke-tierney at uiowa.edu wrote:
#
It's described in ?library; tere is also a post on the R-blog at

https://developer.r-project.org/Blog/public/2019/03/19/managing-search-path-conflicts/index.html

Best,

luke
On Tue, 23 Jun 2020, Kevin R. Coombes wrote:

            

  
    
#
Still, if pkgA imports the generic from pkgB, then installing pkgA installs pkgB and all of its dependencies (Depends and Imports). Which of course makes sense (as otherwise pkgB cannot be installed). But all of this just for importing

foo <- function(x, ...)
   UseMethod("foo")

from pkgB.

Best,
Wolfgang
#
On 23/06/2020 10:28 a.m., Viechtbauer, Wolfgang (SP) wrote:
I think you'd need to be more specific about the two packages before I 
would believe this is much of a problem.

If pkgA and pkgB both contain methods for the same generic, then they 
are probably working in the same problem domain, and already share many 
dependencies.  It seems it would be rare that pkgA's dependencies and 
pkgB's dependencies are both big sets that don't have a lot of overlap.

If it's only pkgB that has the big dependency set, then just put the 
generic in pkgA.  And if you really are in that rare case where they 
both have big non-overlapping dependency sets, then create a tiny pkgC 
to hold the generic.

On the other hand, if both packages were allowed to declare foo as a 
generic, and R should think of it as the same generic, confusion would 
follow:

Think about the case of the filter functions in stats and dplyr.  It's 
not a generic in stats, but obviously could be.  In stats, the name is 
used to talk about linear filtering on a time series. (There are several 
different representations of time series in R, so it might make sense 
for stats::filter to be a generic to allow it to work on all of them.)

In dplyr, the same name is used to describe subsetting a dataset.

Those are both valid uses of the word "filter", but they have nothing to 
do with each other.  It's perfectly reasonable to think that a user 
might want to do both kinds of filtering.  If stats::filter was a 
generic and someone wrote a method for dplyr::filter, clearly a call to 
stats::filter should not use that method. It's even possible that some 
package doing time series analysis in the tidyverse framework would want 
to have methods for both generics.

Duncan Murdoch
#
Yes, but I think my concern may have been bogus. WRE says that even though
the whole package is loaded, it is *not* placed on the search path. Only
exports from the package go on the search path. So if the imported generic
is not then explicitly exported, it would not be seen nor cause a conflict
with another generic of the same name that is exported.

At least that's my understanding. However, this discussion is getting too
esoteric for me, so I'm just going to shut up and leave it to folks who are
more knowledgeable. With apologies for my noise.

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Jun 23, 2020 at 7:29 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote: