Skip to content

Converting a list to a data frame

19 messages · Kevin E. Thorpe, Huzefa Khalil, Eivind K. Dovik +8 more

#
I suspect this is pretty easy, but I'm having trouble figuring it out. 
Basically, I have a list of data frames such as the following example:

list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))

I would like to turn this into  data frame where the list elements are 
essentially rbind'ed together and the element name becomes a new 
variable. For example, I would like to turn the list above into a data 
frame that looks like this:

data.frame(type=c("A","A","B","B"),x=c(1:2,5:6),y=c(3:4,7:8))

Appreciate any pointers.

Kevin
#
Hi Kevin,

There is probably a better way, but it can be done in two steps like this

temp <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))

temp <- lapply(names(temp), function(n, temp) {
  temp[[n]]$type <- n
  return(temp[[n]])
}, temp = temp)

do.call(rbind, temp)



On Wed, May 2, 2018 at 1:11 PM, Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
wrote:

  
  
#
'data.frame':   4 obs. of  3 variables:
 $ type: Factor w/ 2 levels "A","B": 1 1 2 2
 $ x   : int  1 2 5 6
 $ y   : int  3 4 7 8


Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Wed, May 2, 2018 at 10:11 AM, Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
wrote:

  
  
#
On Wed, 2 May 2018, Kevin E. Thorpe wrote:

            
Hi, Kevin.

Here's code that will generate your desired data frame.

# List as provided
thelist <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))
thelist

# Creating the type-vector
type <- c()
for(i in 1:length(thelist)){
   type <- c(type, rep(names(thelist)[i], sapply(thelist, nrow)[i]))
}

# Creating the data frame
df <- data.frame(type, do.call(rbind.data.frame, c(thelist, make.row.names 
= FALSE)))
df


Kind regards,
Eivind K. Dovik
Bergen, NO
#
Another approach:

########
library(tidyr)
L <- list( A = data.frame( x=1:2, y=3:4 )
          , B = data.frame( x=5:6, y=7:8 )
          )
D <- data.frame( Type = names( L )
                , stringsAsFactors = FALSE
                )
D$data <- L
unnest(D, data)
#>   Type x y
#> 1    A 1 3
#> 2    A 2 4
#> 3    B 5 7
#> 4    B 6 8
########
On Wed, 2 May 2018, Eivind K. Dovik wrote:

            
---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live Go...
                                       Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k
#
On 05/02/2018 07:11 PM, Kevin E. Thorpe wrote:
Hi Kevin,

data.table::rbindlist does exactly what you want in a very efficient way:

library(data.table)
dat <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))
rbindlist(dat, idcol = "type")

Regards,
Denes
#
Or add the type column first and then rbind:

x <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))
x2 <- do.call(rbind, lapply(names(x), function(z) 
      data.frame(type=z, dat[[z]])))

----------------------------------------
David L Carlson
Department of Anthropology
Texas A&M University
College Station, TX 77843-4352

-----Original Message-----
From: R-help <r-help-bounces at r-project.org> On Behalf Of William Dunlap via R-help
Sent: Wednesday, May 2, 2018 12:28 PM
To: Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
Cc: R Help Mailing List <r-help at r-project.org>
Subject: Re: [R] Converting a list to a data frame
'data.frame':   4 obs. of  3 variables:
 $ type: Factor w/ 2 levels "A","B": 1 1 2 2
 $ x   : int  1 2 5 6
 $ y   : int  3 4 7 8


Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Wed, May 2, 2018 at 10:11 AM, Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
wrote:
______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
#
Typo: dat[[z]] should be x[[z]]:

x2 <- do.call(rbind, lapply(names(x), function(z) 
      data.frame(type=z, x[[z]])))
x2
  type x y
1    A 1 3
2    A 2 4
3    B 5 7
4    B 6 8

David C

-----Original Message-----
From: R-help <r-help-bounces at r-project.org> On Behalf Of David L Carlson
Sent: Wednesday, May 2, 2018 3:51 PM
To: William Dunlap <wdunlap at tibco.com>; Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
Cc: r-help mailing list <r-help at r-project.org>
Subject: Re: [R] Converting a list to a data frame

Or add the type column first and then rbind:

x <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))
x2 <- do.call(rbind, lapply(names(x), function(z) 
      data.frame(type=z, dat[[z]])))

----------------------------------------
David L Carlson
Department of Anthropology
Texas A&M University
College Station, TX 77843-4352

-----Original Message-----
From: R-help <r-help-bounces at r-project.org> On Behalf Of William Dunlap via R-help
Sent: Wednesday, May 2, 2018 12:28 PM
To: Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
Cc: R Help Mailing List <r-help at r-project.org>
Subject: Re: [R] Converting a list to a data frame
'data.frame':   4 obs. of  3 variables:
 $ type: Factor w/ 2 levels "A","B": 1 1 2 2
 $ x   : int  1 2 5 6
 $ y   : int  3 4 7 8


Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Wed, May 2, 2018 at 10:11 AM, Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
wrote:
______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
#
This is very nice to learn about, Denis, but it seems only fair to point out that the result of rbindlist is not a data frame.  You can convert it to a data frame easily, but the copy and indexing semantics of data tables are quite different than data tables, which could be a real headache for someone not prepared for those differences. (To learn more, read the data tables vignette.)

Tibbles  (as produced by unnest in my previous response) are not data tables either, but they behave much more like data frames than data tables do.
On May 2, 2018 1:30:37 PM MDT, "T?th D?nes" <toth.denes at kogentum.hu> wrote:

  
    
#
On 05/03/2018 07:58 AM, Jeff Newmiller wrote:
In response to Jeff's note, this a solution which results in a 
data.frame instead of a data.table:

library(data.table)
dat <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))
dat <- rbindlist(dat, idcol = "type")

# traditional way
dat_df <- as.data.frame(dat)

# no copy (assignment not needed, memory-efficient)
setDF(dat)

Further amendments would be needed to transform the 'type' variable to a 
factor, if it is required.
#
Before this thread gets carried away to data.table and
tibbleverse (and as nobody else has done so) :

Let me nominate this beautiful solution by
Bill Dunlap and David Carlson to win the prize  with a 10 out 10 grade:

Beautiful use of  do.call() and lapply(), two of the most
versatile and important functions from the base R toolbox.

Congratulations!

Martin Maechler
R Core Team
#
On 05/03/2018 03:59 AM, Martin Maechler wrote:
First of all, thank you to all who responded. I have learned (and will 
learn) about several new things. The data.table/tibbleverse is one of 
those places I have yet to tread but it never seems to be a "good" time 
to learn as generally when these tools would be helpful is when I am 
trying to get something done but don't have the time to recast the 
problem in a different paradigm.

Second, I echo Martin's comment. The combined solution of Bill Dunlap 
and David Carlson is beautifully elegant.

Thanks again,

Kevin

  
    
#
Sometimes one want to "bind" list that have some columns in common and others that you want to include from one but leave as NA for the rows from the other. The rbind.list function from package-plyr is from the pre-tidy/tibble era of Hadley's efforts and provides that facility out of the box. It also seems to "work" with the do.call(rbind strategy for named lists above.

x <- list(A=mtcars[c("mpg", "wt")] , B=mtcars[c("wt", "cyl")])  #mtcars
x2 <- do.call(rbind.fill, lapply(names(x), function(z) 
     data.frame(type=z, x[[z]])))
str(x2)
#---------
'data.frame':	64 obs. of  4 variables:
 type: Factor w/ 2 levels "A","B": 1 1 1 1 1 1 1 1 1 1 ...
 mpg : num  21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
 wt  : num  2.62 2.88 2.32 3.21 3.44 ...
 cyl : num  NA NA NA NA NA NA NA NA NA NA ...
#
On Wed, May 2, 2018 at 11:53 AM, Jeff Newmiller
<jdnewmil at dcn.davis.ca.us> wrote:
I think a slightly more idiomatic tidyverse solution is dplyr::bind_rows()

l <- list(
  A = data.frame(x = 1:2, y = 3:4),
  B = data.frame(x = 5:6, y = 7:8)
)

dplyr::bind_rows(l, .id = "type")
#>   type x y
#> 1    A 1 3
#> 2    A 2 4
#> 3    B 5 7
#> 4    B 6 8

This also has the advantage of returning a data frame when the inputs
are data frames.

Hadley
#
On 05/03/2018 01:28 PM, Hadley Wickham wrote:
I _clearly_ need to learn the dplyr package.
#
If you require that the 'type' column be a factor with a level for each
element of the input list, then you need to do that after calling
dplyr::bind_rows(), just as with the base-R solutions.
Y=integer()), C = data.frame(X=3L, Y=13L) )
'data.frame':   3 obs. of  3 variables:
 $ Which: chr  "A" "A" "C"
 $ X    : int  1 2 3
 $ Y    : int  11 12 13
'data.frame':   3 obs. of  3 variables:
 $ Which: Factor w/ 3 levels "A","B","C": 1 1 3
 $ X    : int  1 2 3
 $ Y    : int  11 12 13

Sometimes you need the names of the of the 0-row data.frames in the output
to make plots, etc., comparable across various samples of the data.


Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Thu, May 3, 2018 at 10:28 AM, Hadley Wickham <h.wickham at gmail.com> wrote:

            

  
  
#
Good morning.

Novice usR. Here.

I am following this string, among many, learning as I go.

Quick question please?

I thought that perhaps ata.frame was part of the zoo pkg, b/c when I searched it came up in help?

However, evidently not or I am not using it properly.

Please advise, thank you.
x <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))
x2 <- do.call(rbind, lapply(names(x), function(z)
ata.frame(type=z, dat[[z]])))
#Error in ata.frame(type = z, dat[[z]]) : could not find function "ata.frame"

?ata.frame
??ata.frame #Looks like it's part of the zoo package?
install.packages("zoo")

#Typo: dat[[z]] should be x[[z]]:

x <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))
x2 <- do.call(rbind, lapply(names(x), function(z)
ata.frame(type=z, x[[z]])))
#Error in ata.frame(type = z, dat[[z]]) : still cannot find function "ata.frame"?

William H. Poling, Ph.D.

From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of Huzefa Khalil
Sent: Wednesday, May 02, 2018 1:24 PM
To: Kevin E. Thorpe <kevin.thorpe at utoronto.ca>
Cc: R Help Mailing List <r-help at r-project.org>
Subject: Re: [R] Converting a list to a data frame

Hi Kevin,

There is probably a better way, but it can be done in two steps like this

temp <- list(A=data.frame(x=1:2, y=3:4),B=data.frame(x=5:6,y=7:8))

temp <- lapply(names(temp), function(n, temp) {
temp[[n]]$type <- n
return(temp[[n]])
}, temp = temp)

do.call(rbind, temp)



On Wed, May 2, 2018 at 1:11 PM, Kevin E. Thorpe <kevin.thorpe at utoronto.ca<mailto:kevin.thorpe at utoronto.ca>>
wrote:
______________________________________________
R-help at r-project.org<mailto:R-help at r-project.org> mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html<http://www.R-project.org/posting-guide.html>
and provide commented, minimal, self-contained, reproducible code.

Confidentiality Notice This message is sent from Zelis. ...{{dropped:15}}
#
It looks like you made a copy/paste error below. Your ata.frame should 
be data.frame.

Kevin
On 05/04/2018 08:18 AM, Bill Poling wrote:

  
    
#
Oh, how funny, hence the term Novice usR. UGH!

Thank you Sir.

WHP



From: Kevin E. Thorpe [mailto:kevin.thorpe at utoronto.ca]
Sent: Friday, May 04, 2018 9:08 AM
To: Bill Poling <Bill.Poling at zelis.com>; Huzefa Khalil <huzefa.khalil at umich.edu>
Cc: R Help Mailing List <r-help at r-project.org>
Subject: Re: [R] Converting a list to a data frame

It looks like you made a copy/paste error below. Your ata.frame should
be data.frame.

Kevin
On 05/04/2018 08:18 AM, Bill Poling wrote:
--
Kevin E. Thorpe
Head of Biostatistics, Applied Health Research Centre (AHRC)
Li Ka Shing Knowledge Institute of St. Michael's Hospital
Assistant Professor, Dalla Lana School of Public Health
University of Toronto
email: kevin.thorpe at utoronto.ca<mailto:kevin.thorpe at utoronto.ca> Tel: 416.864.5776 Fax: 416.864.3016

Confidentiality Notice This message is sent from Zelis. ...{{dropped:15}}