dplyr/summarize does not create a true data frame

Thanks to John Kane for an off-list consultation. As the following annotated transcript shows, it's the group_by() function that transforms a data frame into something else:  a "grouped_df" object that *looks* identical to the original data frame (e.g. the rows are in the original order -- *not* grouped, as arrange() would do), but does not always act like a data frame.

library(dplyr)

# set up data frame, and show its structure [ see below for clean copy of dput() code ]

frm = structure(list(Id = structure(1:10, .Label = c("P01", "P02", 

+ "P03", "P04", "P05", "P06", "P07", "P08", "P09", "P10"), class = "factor"), 
+     Sex = structure(c(2L, 1L, 1L, 2L, 2L, 2L, 1L, 2L, 2L, 1L), .Label = c("Female", 
+     "Male"), class = "factor"), Height = structure(c(1L, 1L, 
+     3L, 2L, 1L, 3L, 1L, 2L, 1L, 1L), .Label = c("Short", "Medium", 
+     "Tall"), class = "factor"), Value = c(69.47, 64.61, 74.77, 
+     73.31, 64.76, 72.78, 64.64, 55.96, 60.45, 51.11)), .Names = c("Id", 
+ "Sex", "Height", "Value"), row.names = c(NA, -10L), class = "data.frame")

str(frm)

'data.frame':	10 obs. of  4 variables:
 $ Id    : Factor w/ 10 levels "P01","P02","P03",..: 1 2 3 4 5 6 7 8 9 10
 $ Sex   : Factor w/ 2 levels "Female","Male": 2 1 1 2 2 2 1 2 2 1
 $ Height: Factor w/ 3 levels "Short","Medium",..: 1 1 3 2 1 3 1 2 1 1
 $ Value : num  69.5 64.6 74.8 73.3 64.8 ...

# run group_by() on data frame, and show resulting structure

after.group_by = frm %>% group_by(Sex, Height)

str(after.group_by)

Classes 'grouped_df', 'tbl_df', 'tbl' and 'data.frame':	10 obs. of  4 variables:
 $ Id    : Factor w/ 10 levels "P01","P02","P03",..: 1 2 3 4 5 6 7 8 9 10
 $ Sex   : Factor w/ 2 levels "Female","Male": 2 1 1 2 2 2 1 2 2 1
 $ Height: Factor w/ 3 levels "Short","Medium",..: 1 1 3 2 1 3 1 2 1 1
 $ Value : num  69.5 64.6 74.8 73.3 64.8 ...
 - attr(*, "vars")=List of 2
  ..$ : symbol Sex
  ..$ : symbol Height
 - attr(*, "drop")= logi TRUE
 - attr(*, "indices")=List of 5
  ..$ : int  1 6 9
  ..$ : int 2
  ..$ : int  0 4 8
  ..$ : int  3 7
  ..$ : int 5
 - attr(*, "group_sizes")= int  3 1 3 2 1
 - attr(*, "biggest_group_size")= int 3
 - attr(*, "labels")='data.frame':	5 obs. of  2 variables:
  ..$ Sex   : Factor w/ 2 levels "Female","Male": 1 1 2 2 2
  ..$ Height: Factor w/ 3 levels "Short","Medium",..: 1 3 1 2 3
  ..- attr(*, "vars")=List of 2
  .. ..$ : symbol Sex
  .. ..$ : symbol Height

# the two data structure *seem* to be the same ...  

frm == after.group_by

Id  Sex Height Value
 [1,] TRUE TRUE   TRUE  TRUE
 [2,] TRUE TRUE   TRUE  TRUE
 [3,] TRUE TRUE   TRUE  TRUE
   ...etc.

# ... but they're not

frm[4]

Value
1  69.47
2  64.61
   ...etc.

after.group_by[4]

Error in eval(expr, envir, enclos) : index out of bounds

# fortunately, we can convert back to a true data frame

as.data.frame(after.group_by)[4]

Value
1  69.47
2  64.61
   ...etc.

################################## dput() code below

structure(list(Id = structure(1:10, .Label = c("P01", "P02", 
"P03", "P04", "P05", "P06", "P07", "P08", "P09", "P10"), class = "factor"), 
    Sex = structure(c(2L, 1L, 1L, 2L, 2L, 2L, 1L, 2L, 2L, 1L), .Label = c("Female", 
    "Male"), class = "factor"), Height = structure(c(1L, 1L, 
    3L, 2L, 1L, 3L, 1L, 2L, 1L, 1L), .Label = c("Short", "Medium", 
    "Tall"), class = "factor"), Value = c(69.47, 64.61, 74.77, 
    73.31, 64.76, 72.78, 64.64, 55.96, 60.45, 51.11)), .Names = c("Id", 
"Sex", "Height", "Value"), row.names = c(NA, -10L), class = "data.frame")

-----Original Message-----
From: John Kane [mailto:jrkrideau at inbox.com]
Sent: Friday, November 21, 2014 12:33 PM
To: John Posner; 'r-help at r-project.org'
Subject: RE: [R] dplyr/summarize does not create a true data frame

Your code in creating 'frm' is not working for me and it is complicated enough
that I don't want to work it out. See ?dput for a better way to supply data.
Also see:
https://github.com/hadley/devtools/wiki/Reproducibility
 http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-
reproducible-example

That said, I don't see why 'my.output[4]' is not working.  Try something like
str(frm) to see what you have there and/or resubmit the data in dput format

See simple example below:

dat1  <- data.frame(aa = sample(1:20, 100, replace = TRUE), bb = 1:100 )
dat1[2]

John Kane
Kingston ON Canada

-----Original Message-----
From: john.posner at mjbiostat.com
Sent: Fri, 21 Nov 2014 17:10:16 +0000
To: r-help at r-project.org
Subject: [R] dplyr/summarize does not create a true data frame

I got an error when trying to extract a 1-column subset of a data
frame (called "my.output") created by dplyr/summarize. The ncol()
function says that my.output has 4 columns, but "my.output[4]" fails.
Note that converting my.output using as.data.frame() makes for a happy

ending.

Is this the intended behavior of dplyr?

Tx,
John

library(dplyr)

# set up data frame
rows = 100
repcnt = 50
sexes = c("Female", "Male")
heights = c("Med", "Short", "Tall")

frm = data.frame(

+   Id = paste("P", sprintf("%04d", 1:rows), sep=""),
+   Sex = sample(rep(sexes, repcnt), rows, replace=T),
+   Height = sample(rep(heights, repcnt), rows, replace=T),
+   V1 = round(runif(rows)*25, 2) + 50,
+   V2 = round(runif(rows)*1000, 2) + 50,
+   V3 = round(runif(rows)*350, 2) - 175
+ )

# use dplyr/summarize to create data frame my.output = frm %>%

+   group_by(Sex, Height) %>%
+   summarize(V1sum=sum(V1), V2sum=sum(V2))

# work with columns in the output data frame
ncol(my.output)

[1] 4

my.output[1]

Source: local data frame [6 x 1]
Groups: Sex

     Sex
1 Female
2 Female
3 Female
4   Male
5   Male
6   Male

my.output[4]

Error in eval(expr, envir, enclos) : index out of bounds  ########
ERROR HERE

as.data.frame(my.output)[4]

     V2sum
1 12427.97
2  8449.82
3  8610.97
4  7249.20
5 12616.91
6 10372.15

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__________________________________________________________
__
FREE ONLINE PHOTOSHARING - Share your photos online with your friends
and family!
Visit http://www.inbox.com/photosharing to find out more!