Enormous Datasets

It depends on what you mean by 'handle', but probably not.  You'll 
likely have to split the file into multiple files unless you have some 
rather high end hardware.   However, in my limited experience, there's 
almost always a meaningful way to split the data (geographically, or 
by other categories).

A few things I've learned recently working with large datasets:

1.  Store files in .rda format using save() -- the load times are much 
faster and loading takes up less memory
2.  If your data are integers, store them as integers!
3.  Don't store character variables in dataframes -- use factors

-roger

Enormous Datasets

Thread (3 messages)