Manage huge database
Hi, You can treat it as a database and use ODBC to fetch data from the CSV file using SQL. See the package RODBC for details about database connections. (I have dealt with similar problems before with RODBC) Regards, Yihui -- Yihui Xie <xieyihui at gmail.com> Phone: +86-(0)10-82509086 Fax: +86-(0)10-82509086 Mobile: +86-15810805877 Homepage: http://www.yihui.name School of Statistics, Room 1037, Mingde Main Building, Renmin University of China, Beijing, 100872, China
On Mon, Sep 22, 2008 at 2:50 PM, Jos? E. Lozano <lozalojo at jcyl.es> wrote:
Hello,
Recently I have been trying to open a huge database with no success.
It's a 4GB csv plain text file with around 2000 rows and over 500,000
columns/variables.
I have try with The SAS System, but it reads only around 5000 columns, no
more. R hangs up when opening.
Is there any way to work with "parts" (a set of columns) of this database,
since its impossible to manage it all at once?
Is there any way to establish a link to the csv file and to state the
columns you want to fetch every time you make an analysis?
I've been searching the net, but found little about this topic.
Best regards,
Jose Lozano
[[alternative HTML version deleted]]