An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20120728/88de713b/attachment.pl>
using save() to work with objects that exceed memory capacity
3 messages · David Romano, R. Michael Weylandt
On Sat, Jul 28, 2012 at 10:48 AM, David Romano <romanod at grinnell.edu> wrote:
Context: I'm relatively new to R and am working with very large datasets. General problem: If working on a dataset requires that I produce more than two objects of roughly the size of the dataset, R quickly uses up its available memory and slows to a virtual halt. My tentative solution: To save and remove objects as they're created, and load them when I need them. To do this I'm trying to automatically generate file names derived from these objects, and use these in save(). My specific question to the list: How do I capture the string that names an object I want to save, in such a way that I can use it in a function that calls save()? For example, suppose I create a matrix and then save it follows:
mat<-matrix(1:9,3,3) save(mat, file="matfile")
Then I get a file of the kind I'd like: the command 'load("matfile")'
retrieves the correct matrix, with the original name 'mat'.
Further, if I instead save it this way:
objectname<-"mat" save(list=ls(pattern=objectname), file="matfile")
then I get the same positive result. But now suppose I create a function
saveobj <- function(objectname,objectfile)
+ {
+ save(list=ls(pattern=objectname),file=objectfile);
+ return()};
Then if I now try to save 'mat' by
matname<-"mat" saveobj(matname,"matfile")
I do not get the same result; namely, the command 'load("mat")' retrieves
no objects. Why is this?
load("matfile") no?
It seems to work for me:
R> x <- matrix(1:9, ncol = 3)
R> saveobj <- function(obj, file){
+ save(list = obj, file = file)
+ }
R> exists("x")
[1] FALSE
R> saveobj("x", "amatrix.rdat")
R> rm(x)
R> load("amatrix.rdat")
R> x
[,1] [,2] [,3]
[1,] 1 4 7
[2,] 2 5 8
[3,] 3 6 9
Cheers,
Michael
I'd be grateful for any help on either my specific questions, or
suggestions of a better ways to address the issue of limited memory.
Thanks,
David Romano
[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20120730/a25b9b00/attachment.pl>