[R-pkg-devel] S3 length method behavior

I've found that it's a very bad idea to provide length or names
methods for just this reason.
Hadley
Hello,

I have run into an issue while developing an R package. Specifically, my
issue relates to what happens when I define an S3 length method for my
class. Here is my basic setup:

# Here is the structure of myClass
# uses an externalptr to operate on file using some C functions
list(file = "my/file/location", handle = <externalptr>)

# this function gets an S3Method() in NAMESPACE
length.myClass <- function(x) {
  .Call("get_class_length",myPkg)
}

As you can see, myClass is just a list of two pieces of information about
the particular object. Without a defined length method, length(myClass)
would be 2. However, I defined a length method to give me the information
about the amount of binary messages contained in the file the object points
to. In one of my test files there are 82 messages and length(myObject)
correctly returns 82. What I ran into is that I was crashing RStudio when
rebuilding my package with an object of myClass in my environment. Only
removing my S3 length method caused the crash to not occur. After looking
for memory leaks and other errors I finally noticed that the str() on the
object of myClass looked odd. It returned something like this:

List of 82
 $ file  : chr "my/file/location"
 $ handle:<externalptr>
 $ NA:
Error in object[[i]] : subscript out of bounds

It seems that when I define an S3 method for length that somehow the object
of myClass then gets restructured to have the length returned by the
function. There are 82 messages in the file, but myClass objects should
only have two elements. My best guess here is that the RStudio crash was
happening because the object was misrepresented internally. Of course, I
cannot be sure. None of my tests pointed to anything useful.

My questions are, then, whether this behavior makes sense and what to do
about it. If I define my own str() method, will that fix it? I think I am
just misunderstanding what is going on with the methods I have defined.
Hopefully, someone can offer some clarity.

Thanks,

        [[alternative HTML version deleted]]

______________________________________________
R-package-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

http://had.co.nz/
I've found that it's a very bad idea to provide length or names
methods for just this reason.
After looking
for memory leaks and other errors I finally noticed that the str() on the
object of myClass looked odd. It returned something like this:

List of 82
 $ file  : chr "my/file/location"
 $ handle:<externalptr>
 $ NA:
Error in object[[i]] : subscript out of bounds
My questions are, then, whether this behavior makes sense and what to do
about it. If I define my own str() method, will that fix it? I think I am
just misunderstanding what is going on with the methods I have defined.
Hopefully, someone can offer some clarity.
Defining a str on your class will at least fix the out of bounds error:

Create a trivial S3 class:

 > z=list(1,22)
 > class(z)="foo"

length method looks at the second element:

 > length.foo=function(x){x[[2]]}
 > length(z)
[1] 22

 and str barfs:

 > str(z)
List of 22
 $ : num 1
 $ : num 22
 $ :Error in object[[i]] : subscript out of bounds

Define a str method:

 > str.foo=function(object,...){for(i in
1:length(unclass(object))){str(unclass(object[[i]]))}}
 > str(z)
 num 1
 num 22

BUT... the real problem is that S3 classes are seriously informal and
there's no concept of what methods you need to define on a class
because there's no concept of an "interface" that new classes have to
conform to. So stuff breaks, seemingly at random, and via action at a
distance. Somewhere something is going to expect z[[1]] to
z[[length(z)]] to exist, which is what the default str is doing...

+1 on Hadley - don't override any basic R structural methods, create
new ones with new names. You can make them more meaningful too. For
your example, maybe "messageCount(myObject)"?

Barry
After toying around with trying some of the suggested fixes, I have to
agree that it's more trouble than it is worth to add my own primitive
methods. It'll work just fine to move on and write my own methods. Thanks
to all for the explanations and suggestions.

On Tue, Feb 2, 2016 at 11:23 AM, Barry Rowlingson <

On Tue, Feb 2, 2016 at 3:28 PM, Hadley Wickham <h.wickham at gmail.com>
wrote:
I've found that it's a very bad idea to provide length or names
methods for just this reason.

After looking
for memory leaks and other errors I finally noticed that the str() on
the
object of myClass looked odd. It returned something like this:

List of 82
 $ file  : chr "my/file/location"
 $ handle:<externalptr>
 $ NA:
Error in object[[i]] : subscript out of bounds

My questions are, then, whether this behavior makes sense and what to do
about it. If I define my own str() method, will that fix it? I think I
am
just misunderstanding what is going on with the methods I have defined.
Hopefully, someone can offer some clarity.
 Defining a str on your class will at least fix the out of bounds error:

Create a trivial S3 class:

 > z=list(1,22)
 > class(z)="foo"
length method looks at the second element:

 > length.foo=function(x){x[[2]]}
 > length(z)
[1] 22

 and str barfs:

 > str(z)
List of 22
 $ : num 1
 $ : num 22
 $ :Error in object[[i]] : subscript out of bounds

Define a str method:

 > str.foo=function(object,...){for(i in
1:length(unclass(object))){str(unclass(object[[i]]))}}
 > str(z)
 num 1
 num 22

BUT... the real problem is that S3 classes are seriously informal and
there's no concept of what methods you need to define on a class
because there's no concept of an "interface" that new classes have to
conform to. So stuff breaks, seemingly at random, and via action at a
distance. Somewhere something is going to expect z[[1]] to
z[[length(z)]] to exist, which is what the default str is doing...

+1 on Hadley - don't override any basic R structural methods, create
new ones with new names. You can make them more meaningful too. For
your example, maybe "messageCount(myObject)"?

Barry

Barry Rowlingson <b.rowlingson at lancaster.ac.uk>
    on Tue, 2 Feb 2016 17:23:46 +0000 writes:

>> I've found that it's a very bad idea to provide length or names
    >> methods for just this reason.

well, not quite, see below ..

    >>> After looking
    >>> for memory leaks and other errors I finally noticed that the str() on the
    >>> object of myClass looked odd. It returned something like this:
    >>> 
    >>> List of 82
    >>> $ file  : chr "my/file/location"
    >>> $ handle:<externalptr>
    >>> $ NA:
    >>> Error in object[[i]] : subscript out of bounds

    >>> My questions are, then, whether this behavior makes sense and what to do
    >>> about it. If I define my own str() method, will that fix it? I think I am
    >>> just misunderstanding what is going on with the methods I have defined.
    >>> Hopefully, someone can offer some clarity.

    > Defining a str on your class will at least fix the out of bounds error:

    > Create a trivial S3 class:

    >> z=list(1,22)
    >> class(z)="foo"

    > length method looks at the second element:

    >> length.foo=function(x){x[[2]]}
    >> length(z)
    > [1] 22

    > and str barfs:

    >> str(z)
    > List of 22
    > $ : num 1
    > $ : num 22
    > $ :Error in object[[i]] : subscript out of bounds

    > Define a str method:

    >> str.foo=function(object,...){for(i in
    > 1:length(unclass(object))){str(unclass(object[[i]]))}}
    >> str(z)
    > num 1
    > num 22

    > BUT... the real problem is that S3 classes are seriously informal and
    > there's no concept of what methods you need to define on a class
    > because there's no concept of an "interface" that new classes have to
    > conform to. So stuff breaks, seemingly at random, and via action at a
    > distance. Somewhere something is going to expect z[[1]] to
    > z[[length(z)]] to exist, which is what the default str is doing...

Indeed.
Still, it can also be advantageous to define such methods *consistently*.

With *consistence*, I mean that at least

- names(obj) either returns NULL or a character vector of
  length length(obj), and that as Barry mentions,
- obj[[ i ]]  is meaningful  for (i in  seq_along(obj))
           [yes, seq_along(.) automatically works with your length() method !]

- often you'd also want  obj[ i ]  to also work consistently
  (sometimes identically to `[[`)

I'd say that oftentimes it may be easier (and more "rewarding")
to define such `[` and `[[` methods for your class anyway.

As author of str(), I'll declare the design(*) of str() to be such
that with these methods (length, names, `[`, `[[`) defined
consistently, str.default(obj) already works sensibly.  
The alternative is indeed to define your own str() method.  

One of the two you'd want often, because e.g.,
  str( list( <obj1>, <obj2> ) )
or similar things should work too.

--
(*) to be honest, str() grew and developed very much
    historically, so the above is more an "implementation principle"

    > +1 on Hadley - don't override any basic R structural methods, create
    > new ones with new names. You can make them more meaningful too. For
    > your example, maybe "messageCount(myObject)"?

    > Barry