Skip to content

R does in memory analysis only?

2 messages · Ross Boylan, Duncan Murdoch

#
I wonder if someone would confirm something I'm 99% sure of from the
docs and discussion on the list, but can't find stated explicitly:
R works only on problems that fit into (real or virtual) memory.

Thus, even if you have a problem (e.g., simple regression) that could be
solved by doing some operation on each row of a dataset at a time, you
can't solve it unless the entire dataset and associated intermediate
results fit in memory.

So if you're in 32 bits, your max problem size is about 2G (regular
Windows and Linux limit your process size to this, though I think some
fancy versions let you go a bit higher).

Is there any thought of relaxing this limitation?  I realize doing so
would be a big job.  I also realize that 64 bits makes it much less
pressing.

Finally, does S-Plus have the same limitation?

Thanks.
#
On Sat, 07 Feb 2004 14:15:54 -0800, you wrote:

            
The base libraries have that limitation.  I think various people are
working on packages that handle objects that don't fit in memory.
I haven't heard of any plans to remove the limit in the base packages.
There's definitely interest in supporting 64 bit platforms.

Duncan Murdoch