Wrong length of POSIXt vectors (PR#10507)

"TP" == Tony Plate <tplate at acm.org>
    on Fri, 14 Dec 2007 13:58:30 -0700 writes:

   TP> Duncan Murdoch wrote:

   >> On 12/13/2007 1:59 PM, Tony Plate wrote:

   >>> Duncan Murdoch wrote:

   >>>> On 12/11/2007 6:20 AM, simecek at gmail.com wrote:

   >>>>> Full_Name: Petr Simecek
   >>>>> Version: 2.5.1, 2.6.1
   >>>>> OS: Windows XP
   >>>>> Submission from: (NULL) (195.113.231.2)
   >>>>>
   >>>>>
   >>>>> Several times I have experienced that a length of a POSIXt vector
   >>>>> has not been
   >>>>> computed right.
   >>>>>
   >>>>> Example:
   >>>>>
   >>>>> tv<-structure(list(sec = c(50, 0, 55, 12, 2, 0, 37, NA, 17, 3, 31
   >>>>> ), min = c(1L, 10L, 11L, 15L, 16L, 18L, 18L, NA, 20L, 22L, 22L
   >>>>> ), hour = c(12L, 12L, 12L, 12L, 12L, 12L, 12L, NA, 12L, 12L, 12L),
   >>>>> mday = c(13L, 13L, 13L, 13L, 13L, 13L, 13L, NA, 13L, 13L, 13L), mon
   >>>>> = c(5L, 5L, 5L, 5L, 5L, 5L, 5L, NA, 5L, 5L, 5L), year = c(105L,
   >>>>> 105L, 105L, 105L, 105L, 105L, 105L, NA, 105L, 105L, 105L), wday =
   >>>>> c(1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, 1L, 1L, 1L), yday = c(163L, 163L,
   >>>>> 163L, 163L, 163L, 163L, 163L, NA, 163L, 163L, 163L), isdst = c(1L,
   >>>>> 1L, 1L, 1L, 1L, 1L, 1L, -1L, 1L, 1L, 1L)), .Names = c("sec", "min",
   >>>>> "hour", "mday", "mon", "year", "wday", "yday", "isdst"
   >>>>> ), class = c("POSIXt", "POSIXlt"))
   >>>>>
   >>>>> print(tv)
   >>>>> # print 11 time points (right)
   >>>>>
   >>>>> length(tv)
   >>>>> # returns 9 (wrong)

   >>>>
   >>>> tv is a list of length 9.  The answer is right, your expectation is
   >>>> wrong.

   >>>>> I have tried that on several computers with/without switching to
   >>>>> English
   >>>>> locales, i.e. Sys.setlocale("LC_TIME", "en"). I have searched a
   >>>>> help pages but I
   >>>>> cannot imagine how that could be OK.

   >>>>
   >>>> See this in ?POSIXt:
   >>>>
   >>>> Class '"POSIXlt"' is a named list of vectors...
   >>>>
   >>>> You could define your own length measurement as
   >>>>
   >>>> length.POSIXlt <- function(x) length(x$sec)
   >>>>
   >>>> and you'll get the answer you expect, but be aware that length.XXX
   >>>> methods are quite rare, and you may surprise some of your users.
   >>>>

   >>>
   >>> On the other hand, isn't the fact that length() currently always
   >>> returns 9 for POSIXlt objects likely to be a surprise to many users
   >>> of POSIXlt?
   >>>
   >>> The back of "The New S Language" says "Easy-to-use facilities allow
   >>> you to organize, store and retrieve all sorts of data. ... S
   >>> functions and data organization make applications easy to write."
   >>>
   >>> Now, POSIXlt has methods for c() and vector subsetting "[" (and many
   >>> other vector-manipulation methods - see methods(class="POSIXlt")).
   >>> Hence, from the point of view of intending to supply "easy-to-use
   >>> facilities ... [for] all sorts of data", isn't it a little
   >>> incongruous that length() is not also provided -- as 3 functions (any
   >>> others?) comprise a core set of vector-manipulation functions?
   >>>
   >>> Would it make sense to have an informal prescription (e.g., in
   >>> R-exts) that a class that implements a vector-like object and
   >>> provides at least of one of functions 'c', '[' and 'length' should
   >>> provide all three?  It would also be easy to describe a test-suite
   >>> that should be included in the 'test' directory of a package
   >>> implementing such a class, that had some tests of the basic
   >>> vector-manipulation functionality, such as:
   >>>

   >>> > # at this point, x0, x1, x3, & x10 should exist, as vectors of the
   >>> > # class being tested, of length 0, 1, 3, and 10, and they should
   >>> > # contain no duplicate elements
   >>> > length(x0)

   >>> [1] 1

   >>> > length(c(x0, x1))

   >>> [1] 2

   >>> > length(c(x1,x10))

   >>> [1] 11

   >>> > all(x3 == x3[seq(len=length(x3))])

   >>> [1] TRUE

   >>> > all(x3 == c(x3[1], x3[2], x3[3]))

   >>> [1] TRUE

   >>> > length(c(x3[2], x10[5:7]))

   >>> [1] 4

   >>> >

   >>>
   >>> It would also be possible to describe a larger set of vector
   >>> manipulation functions that should be implemented together, including
   >>> e.g., 'rep', 'unique', 'duplicated', '==', 'sort', '[<-', 'is.na',
   >>> head, tail ... (many of which are provided for POSIXlt).
   >>>
   >>> Or is there some good reason that length() cannot be provided (while
   >>> 'c' and '[' can) for some vector-like classes such as "POSIXlt"?

   >>
   >> What you say sounds good in general, but the devil is in the details.
   >> Changing the meaning of length(x) for some objects has fairly
   >> widespread effects.  Are they all positive?  I don't know.
   >>
   >> Adding a prescription like the one you suggest would be good if it's
   >> easy to implement, but bad if it's already widely violated.  How many
   >> base or CRAN or Bioconductor packages violate it currently?   Do the
   >> ones that provide all 3 methods do so in a consistent way, i.e. does
   >> "length(x)" mean the same thing in all of them?

   TP> I'm not sure doing something like this would be so bad even if it is
   TP> already widely violated.  R has evolved significantly over time, and
   TP> many rough edges have been cleaned up, sometimes in ways that were not
   TP> backward compatible.  This is a great thing & my thanks go to the people
   TP> working on R.

   TP> If some base or CRAN or Bioconductor packages currently don't implement
   TP> vector operations consistently, wouldn't it be good to know that?
   TP> Wouldn't it be useful to have an automatic way of determining whether a
   TP> particular vector-like class is consistent with generally agreed set of
   TP> principles for how basic vector operations should work -- things like
   TP> length(x)+length(y)==length(c(x,y))?  This could help developers check,
   TP> document & improve their code, and it could help users understand how to
   TP> use a class, and to evaluate the software quality of a class
   TP> implementation and whether or not it provides the functionality they need.

   >> I agree that the current state is less than perfect, but making it
   >> better would really be a lot of work.  I suspect there are better ways
   >> to spend my time, so I'm not going to volunteer to do it.  I'm not
   >> even going to invite someone else to do it, or offer to review your
   >> work if you volunteer.  I think this falls into the class of "next
   >> time we write a language, let's handle this better" problems.

   TP> Thanks very much for the thoughtful (and honest) feedback!  I suspect
   TP> that the current state could be improved with just a little work, and
   TP> without forcing anyone to do any work they don't want to do.  I'll think
   TP> about this more and try to come back with a better & more concrete
   TP> suggestion.

Good. From "the outside" (i.e. superficial gut feeling :-)
I've sympathized with your suggestion, Tony, quite a bit.
Further, my own taste would probably also have lead me to define
length.POSIXlt differently ..
OTOH, I agree with Duncan that it may be too late to change it
and even more to enforce the consistency rules you propose.
If with a small bit of code (and some patience) we could check
all of CRAN and hopefully bioconductor packages and find only a
very few where it was violated, the whole endeavor may be worth it
... for the sake of making  R more consistent, easier to teach, etc..

Unfortunately I don't remember now what happened many months ago
when I indeed did experiment with having something like

 length.POSIXlt <- function(x) length(x$sec)

Martin Maechler