Reading extremly large comma separated files?
Are there any tricks or wise suggestions on how to import huge comma separated files into R? We have used the ordinary "read syntax" but it takes forever....
One potential time- and memory saver is to specify the class of each column using colClasses in read.table.
For a large data set (800 000 rows) on Windows I also found it quicker and more reliable to import the data into a MySQL database and then use ODBC to read the data into R than to use read.table on the raw data - but that was in part because I hada bit of trouble with the file encoding and MySQL/ODBC sidestepped the issue.
S Ellison
*******************************************************************
This email and any attachments are confidential. Any use...{{dropped:8}}