Have you looked at the g.data package? It might be useful (but may still require some redesign of your dataset). Greg Snow, Ph.D. Statistical Data Center, LDS Hospital Intermountain Health Care greg.snow at ihc.com (801) 408-8111
"Ken Termiso" <jerk_alert at hotmail.com> 10/13/05 08:14AM >>>
I'd put the extra columns in their own data frame, and save that to
disk
(use dates/times/process ids or some other unique identifier in the filenames to distinguish them). When you need access to a mixture of
columns, load (or source, depending how you did the save) the columns
you
need, and cbind them together into one big data frame. If you are concerned about memory requirements when producing the
pieces,
watch out that you don't write out so much data that you'll never have
enough memory to load all you need at once. Duncan Murdoch
hmm...maybe i should just be dumping to a text file instead of a data frame..is there any way (without using a real SQL database) in R to create a file that i can selectively load certain columns from? if not, maybe i should break the data frame up into pieces (as you suggested) and create a separate file that keeps track of which columns are stored in which files (like a hashtable) and just load the small file of keys each time i need to load something.. whaddya think?? ______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html