Corrupt data frame construction - bug?

Thanks Duncan,

Comments and a proposed bug fix in-line below:

-----Original Message-----
From: Duncan Murdoch [mailto:murdoch at stats.uwo.ca]
Sent: Wednesday, April 29, 2009 5:10 PM
To: Steven McKinney
Cc: R-help at r-project.org
Subject: Re: [R] Corrupt data frame construction - bug?

On 29/04/2009 6:41 PM, Steven McKinney wrote:

Hi useRs,

A recent coding infelicity along these lines yielded a corrupt data
frame.

foo <- matrix(1:12, nrow = 3)
bar <- data.frame(foo)
bar$NewCol <- foo[foo[, 1] == 4, 4]
bar
lapply(bar, length)

foo <- matrix(1:12, nrow = 3)
bar <- data.frame(foo)
bar$NewCol <- foo[foo[, 1] == 4, 4]
bar

  X1 X2 X3 X4 NewCol
1  1  4  7 10   <NA>
2  2  5  8 11   <NA>
3  3  6  9 12   <NA>
Warning message:
In format.data.frame(x, digits = digits, na.encode = FALSE) :
  corrupt data frame: columns will be truncated or padded with NAs

lapply(bar, length)

$X1
[1] 3

$X2
[1] 3

$X3
[1] 3

$X4
[1] 3

$NewCol
[1] 0


Is this a bug in the data.frame machinery?
If an attempt is made to add a new column to a data frame, and the

new

object does not have length = number of rows of data frame, or

cannot

be made to have such length via recycling, shouldn't an error be
thrown?

Instead in this example I end up with a "corrupt data frame" having
one zero-length column.


Should this be reported as a bug, or did I misinterpret the
documentation?

I don't think "$" uses any data.frame machinery.  You are working at a
lower level.

If you had added the new column using

bar <- data.frame(bar, NewCol=foo[foo[, 1] == 4, 4])

you would have seen the error:

Error in data.frame(bar, NewCol = foo[foo[, 1] == 4, 4]) :
   arguments imply differing number of rows: 3, 0

But since you treated it as a list, it let you go ahead and create
something that was labelled as a data.frame but wasn't.  This is one

of

the reasons some people prefer S4 methods:  it's easier to protect
against people who mislabel things.

I did some more digging on '$' - there is a data.frame method for it:

getAnywhere("$<-.data.frame" )

A single object matching '$<-.data.frame' was found
It was found in the following places
  package:base
  registered S3 method for $<- from namespace base
  namespace:base
with value

function (x, i, value) 
{
    cl <- oldClass(x)
    class(x) <- NULL
    nrows <- .row_names_info(x, 2L)
    if (!is.null(value)) {
        N <- NROW(value)
        if (N > nrows) 
            stop(gettextf("replacement has %d rows, data has %d", 
                N, nrows), domain = NA)
        if (N < nrows && N > 0L) 
            if (nrows%%N == 0L && length(dim(value)) <= 1L) 
                value <- rep(value, length.out = nrows)
            else stop(gettextf("replacement has %d rows, data has %d", 
                N, nrows), domain = NA)
        if (is.atomic(value)) 
            names(value) <- NULL
    }
    x[[i]] <- value
    class(x) <- cl
    return(x)
}<environment: namespace:base>


I placed a browser() command before return(x) and did some poking
around.

It seems to me there's a bug in this function.  It should be able to
detect the problem I threw at it, and throw an error as you point out is
thrown by the other data.frame assign method.


I modified the rows
          if (N < nrows && N > 0L) 
            if (nrows%%N == 0L && length(dim(value)) <= 1L)
to read
           if (N < nrows) 
            if (N > 0L && nrows%%N == 0L && length(dim(value)) <= 1L)

as in

"$<-.data.frame" <-
function (x, i, value) 
{
    cl <- oldClass(x)
    class(x) <- NULL
    nrows <- .row_names_info(x, 2L)
    if (!is.null(value)) {
        N <- NROW(value)
        if (N > nrows) 
            stop(gettextf("replacement has %d rows, data has %d", 
                N, nrows), domain = NA)
        if (N < nrows) 
            if (N > 0L && nrows%%N == 0L && length(dim(value)) <= 1L) 
                value <- rep(value, length.out = nrows)
            else stop(gettextf("replacement has %d rows, data has %d", 
                N, nrows), domain = NA)
        if (is.atomic(value)) 
            names(value) <- NULL
    }
    x[[i]] <- value
    class(x) <- cl
    return(x)
} 

Now it detects the problem I created, in the fashion you demonstrated
above for the replacement using data.frame().

foo <- matrix(1:12, nrow = 3)
bar <- data.frame(foo)
bar$NewCol <- foo[foo[, 1] == 4, 4]

Error in `$<-.data.frame`(`*tmp*`, "NewCol", value = integer(0)) : 
  replacement has 0 rows, data has 3

It doesn't appear to stumble on weird data frames (these from the
?data.frame help page)

L3 <- LETTERS[1:3]
(d <- data.frame(cbind(x=1, y=1:10), fac=sample(L3, 10,

replace=TRUE)))

(d0  <- d[, FALSE]) # NULL data frame with 10 rows

(d.0 <- d[FALSE, ]) # <0 rows> data frame  (3 cols)

(d00 <- d0[FALSE,])  # NULL data frame with 0 rows

d0$NewCol <- foo[foo[, 1] == 4, 4]

Error in `$<-.data.frame`(`*tmp*`, "NewCol", value = integer(0)) : 
  replacement has 0 rows, data has 10

### Catches this problem above alright.

d.0$NewCol <- foo[foo[, 1] == 4, 4]
d.0

[1] x      y      fac    NewCol
<0 rows> (or 0-length row.names)

### Lets the above one through alright.

d00$NewCol <- foo[foo[, 1] == 4, 4]

d00

[1] NewCol
<0 rows> (or 0-length row.names)
### Lets the above one through alright.


Would the above modification work to fix this problem?

Duncan Murdoch

sessionInfo()

R version 2.9.0 (2009-04-17)
powerpc-apple-darwin8.11.1

locale:
en_CA.UTF-8/en_CA.UTF-8/C/C/en_CA.UTF-8/en_CA.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] nlme_3.1-90

loaded via a namespace (and not attached):
[1] grid_2.9.0      lattice_0.17-22 tools_2.9.0


Also occurs on Windows box with R 2.8.1



Steven McKinney

Statistician
Molecular Oncology and Breast Cancer Program British Columbia Cancer
Research Centre

email: smckinney +at+ bccrc +dot+ ca

tel: 604-675-8000 x7561

BCCRC
Molecular Oncology
675 West 10th Ave, Floor 4
Vancouver B.C.
V5Z 1L3
Canada