Skip to content

S3 lookup rules changed in R 3.6.1

3 messages · Duncan Murdoch, Konrad Rudolph

Original
Konrad Rudolph
# 
tl;dr: S3 lookup no longer works in custom non-namespace environments as of
R 3.6.1. Is this a bug?

I am implementing S3 dispatch for generic methods in environments that are
not
packages. I am trying to emulate the R package namespace mechanism by
having a
?namespace? environment that defines generics and methods, but only exposes
the
generics themselves, not the methods.

To make S3 lookup work when using the generics, I am using
`registerS3method`.
While this method itself has no extensive documentation, the documentation
of
`UseMethod` contains this relevant passage:
registration
does
generic is
the
before
This used to work but it stopped working in R 3.6.1 and I cannot figure out
(a)
why, and (b) how to fix it. Unfortunately I am unable to find the relevant
information by reading the R source code, even when ?diff?ing what seem to
be
the only even remotely relevant changes [1].

The R NEWS merely list the following change for R 3.6.0:
delayed
Unfortunately it is not clear to me what exactly this means.

Here?s a minimal example code that works under R 3.5.3 but breaks under
R 3.6.1
(I don?t know about 3.6.0).

```
# Define ?package namespace?:
ns = new.env(parent = .BaseNamespaceEnv)
local(envir = ns, {
    test = function (x) UseMethod('test')
    test.default = function (x) message('test.default')
    test.foo = function (x) message('test.foo')

    .__S3MethodsTable__. = new.env(parent = .BaseNamespaceEnv)
    .__S3MethodsTable__.$test.default = test.default
    .__S3MethodsTable__.$test.foo = test.foo

    # Or, equivalently:
    # registerS3method('test', 'default', test.default)
    # registerS3method('test', 'foo', test.foo)
})

# Expose generic publicly:
test = ns$test

# Usage:
test(1)
test(structure(1, class = 'foo'))
```

Output in R up to 3.5.3:

```
test.default
test.foo
```

Output in R 3.6.1:

```
Error in UseMethod("test") :
  no applicable method for 'test' applied to an object of class
"c('double', 'numeric')"
```

It?s worth noting that the output of `.S3methods` is the same for all R
versions, and from my understanding of its output, this *should* indicate
that
S3 lookup should behave identically, too. Furthermore, lookup via
`getS3method`
succeeds in all R versions, and (again, in my understanding) the logic of
this
function should be identical to the logic of R?s internal S3 dispatch:

```
getS3method('test', 'default')(1)
getS3method('test', 'foo')(1)
```

Conversely, specialising an existing generic from a loaded package works.
E.g.:

```
local(envir = ns, {
    print.foo = function (x) message('print.foo')
    registerS3method('print', 'foo', print.foo)
})

print(structure(1, class = 'foo'))
```

This prints ?print.foo? in all R versions as expected.

So my question is: Why do the `test(?)` calls in R 3.6.1 no longer trigger
S3
method lookup in the generic function?s environment? Is this behaviour by
design
or is it a bug? If it?s by design, why does `getS3method` still use the old
behaviour? And, most importantly, how can I fix my definition of `ns` to
make
S3 dispatch for non-exposed methods work again?

? actually I just found a workaround:

```
ns$.packageName = 'not important'
```

This marks `ns` as a package namespace. To me, the documentation seems to
imply
that this shouldn?t be necessary (and it previously wasn?t). Furthermore,
the
code for `registerS3method` explicitly supports non-package namespace
environments. Unfortunately this workaround is not satisfactory because
pretending that the environment is a package namespace, when it really
isn?t,
might break other things.

[1] See r75273; there?s also r74625, which changes the actual lookup
mechanism
    used by `UseMethod`, but that seems even less relevant, because it is
    disabled unless a specific environment variable is set.
Duncan Murdoch
#
On 09/10/2019 3:22 p.m., Konrad Rudolph wrote:
I don't know whether this was intentional or not, but a binary search 
through the svn commits finds that the errors started in this one:

------------------------------------------------------------------------
r75127 | hornik | 2018-08-13 09:58:47 -0400 (Mon, 13 Aug 2018) | 2 lines
Changed paths:
    M /trunk/src/main/objects.c
    M /trunk/tests/reg-tests-1a.R

Have S3 methods lookup by default look for the S3 registry in the topenv
of the generic.
------------------------------------------------------------------------

Duncan Murdoch
Konrad Rudolph
# 
Oh, I had missed that that code path is now enabled by default. It?s worth
noting that the commented-out test in that commit also still succeeds if
invoked via `getS3method`. So at the very least there?s now an
inconsistency in the lookup performed by R internally (via `UseMethod`) and
`getS3method`, which is probably unintentional.

I see how the change is beneficial by preventing surprising behaviour in a
corner case. Unfortunately it also breaks at least one published package
[1], and if I understand correctly it no longer conforms to the documented
behaviour (quoted in my initial message), which even explicitly mentions
non-namespace environments.

[1] https://github.com/klmr/modules/issues/147

On Wed, Oct 9, 2019 at 11:23 PM Duncan Murdoch <murdoch.duncan at gmail.com>
wrote: