Large Dataset

Hi Simons,

Is SAS more powerfull than R?

Well, I think I cannot afford to buy SAS.

actually, my computer isn't  really slow. I think 4GB RAM is big enough 
for
personal PC.  I am just wondering, why R running so slow with these specs 
to
handling 3 GB data set. What if the data set were 1 TB?mmm..


Edwin

Hi,

I am not very knowledgeable about this kind of stuff but my guess is that
if you have a fairly slow computer and massive data sets there isnt alot
you can do except get a better computer, buy more RAM or use something 
like
SAS instead?

Hopefully someone else will chip in Edwin, best of luck.

Simon.


----- Original Message -----
From: "Edwin Sendjaja" <edwin7 at web.de>
To: "Simon Pickett" <simon.pickett at bto.org>
Cc: <r-help at r-project.org>
Sent: Tuesday, January 06, 2009 2:53 PM
Subject: Re: [R] Large Dataset

Hi Simon,

My RAM is only 3.2 GB (actually it should be 4 GB, but my Motherboard
doesnt
support it.

R use almost of all my RAM and half of my swap. I think memory.limit 
will
not
solve my problem.  It seems that I need  RAM.

Unfortunately, I can't buy more RAM.

Why R is slow reading big data set?


Edwin

Only a couple of weeks ago I had to deal with this.

adjust the memory limit as follows, although you might not want 4000,
that
is quite high....

memory.limit(size = 4000)

Simon.

----- Original Message -----
From: "Edwin Sendjaja" <edwin7 at web.de>
To: "Simon Pickett" <simon.pickett at bto.org>
Cc: <r-help at r-project.org>
Sent: Tuesday, January 06, 2009 12:24 PM
Subject: Re: [R] Large Dataset

Hi Simon,

Thank for your reply.
I have read ?Memory but I dont understand how to use. I am not sure 
if
that
can solve my problem. Can you tell me more detail?

Thanks,

Edwin

type

?memory

into R and that will explain what to do...

S
----- Original Message -----
From: "Edwin Sendjaja" <edwin7 at web.de>
To: <r-help at r-project.org>
Sent: Tuesday, January 06, 2009 11:41 AM
Subject: [R] Large Dataset

Hi alI,

I  have a 3.1 GB Dataset ( with  11 coloumns and lots data in int
and
string).
If I use read.table; it takes very long. It seems that my RAM is
not big
enough (overload) I have 3.2 RAM and  7GB SWAP, 64 Bit Ubuntu.

Is there a best sultion to read a large data R? I have seen, that
people
suggest to use bigmemory package, ff. But it seems very
complicated. I dont
know how to start with that packages.

i have tried to use bigmemory. But I got some kind of errors. 
Then
I
gave up.


can someone give me an simple example how ot use ff or 
bigmemory?or
maybe
re
better sollution?



Thank you in advance,


Edwin