As I am sitting here waiting for some R scripts to run...I was
wondering... is there any way to parallelize rbind in R?
I wait for this call to complete frequently as I deal with large
amounts of data.
do.call("rbind", LIST)
parallel rbind
5 messages · Steven Bauer, jim holtman, William Dunlap +1 more
What exactly do you mean by "parallelize"? What is wrong with the approach that you are using now? What is a "large amount of data"? Can you give some specifics on the problem you are trying to solve and why your present approach does not appear to be working? What are your expectations of a potential solution?
On Sun, Aug 28, 2011 at 8:36 PM, Steven Bauer <steven.bauer at gmail.com> wrote:
As I am sitting here waiting for some R scripts to run...I was
wondering... is there any way to parallelize rbind in R?
I wait for this call to complete frequently as I deal with large
amounts of data.
do.call("rbind", LIST)
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Jim Holtman Data Munger Guru What is the problem that you are trying to solve?
If you know much about what the elements of LIST look like you can speed things up by not making R figure out what you already know. E.g., if you know that LIST consists of p numeric vectors, all of the same length, n, then the following might be faster matrix(unlist(LIST, use.names=FALSE), nrow=n) If you are worried about row or column names then you can add that information to the call to matrix(). (The above will also work if LIST contains some matrix elements, as long as they all have n rows.) Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Steven Bauer
Sent: Sunday, August 28, 2011 5:36 PM
To: r-help at r-project.org
Subject: [R] parallel rbind
As I am sitting here waiting for some R scripts to run...I was
wondering... is there any way to parallelize rbind in R?
I wait for this call to complete frequently as I deal with large
amounts of data.
do.call("rbind", LIST)
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Oops, I mixed up rbind and cbind. If LIST consists of n numeric vectors, each of length p, try matrix(unlilst(LIST, use.names=FALSE), nrow=n, byrow=TRUE) Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com
-----Original Message----- From: William Dunlap Sent: Sunday, August 28, 2011 6:57 PM To: 'Steven Bauer'; r-help at r-project.org Subject: RE: [R] parallel rbind If you know much about what the elements of LIST look like you can speed things up by not making R figure out what you already know. E.g., if you know that LIST consists of p numeric vectors, all of the same length, n, then the following might be faster matrix(unlist(LIST, use.names=FALSE), nrow=n) If you are worried about row or column names then you can add that information to the call to matrix(). (The above will also work if LIST contains some matrix elements, as long as they all have n rows.) Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Steven Bauer
Sent: Sunday, August 28, 2011 5:36 PM
To: r-help at r-project.org
Subject: [R] parallel rbind
As I am sitting here waiting for some R scripts to run...I was
wondering... is there any way to parallelize rbind in R?
I wait for this call to complete frequently as I deal with large
amounts of data.
do.call("rbind", LIST)
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Steven Bauer <steven.bauer <at> gmail.com> writes:
As I am sitting here waiting for some R scripts to run...I was
wondering... is there any way to parallelize rbind in R?
I wait for this call to complete frequently as I deal with large
amounts of data.
do.call("rbind", LIST)
Perfectly reasonable question, but please don't cross-post
to StackOverflow and R-help -- pick one or the other.
cheers
Ben Bolker