high RAM on Linux or Solaris platform
Dr. Lumley and Prof. Ripley, Thank you very much for your helpful responses. Have you found any particular distribution of Linux to work well with 64-bit R? For the cluster, I am currently considering Debian (since it seems popular) and SUSE (since Matlab runs on it), but I remain open to others. Best regards, David -----Original Message----- From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk] Sent: Tuesday, October 30, 2007 4:51 PM To: Thomas Lumley Cc: David Bickel; r-help at stat.math.ethz.ch Subject: Re: [R] high RAM on Linux or Solaris platform
On Tue, 30 Oct 2007, Thomas Lumley wrote:
On Tue, 30 Oct 2007, David Bickel wrote:
To help me make choices regarding a platform for running high-memory
R
processes in parallel, I would appreciate any responses to these questions: 1. Does the amount of RAM available to an R session depend on the processor (Intel vs. Sun) or on the OS (various Linux distributions
vs.
Solaris)?
Yes. It depends on whether R uses 64-bit or 32-bit pointers. For 64-bit R
you
need a 64-bit processor, an operating system that will run 64-bit programs, and a compiler that will produce them. I'm not sure what the current Intel offerings are, but you can compile
and run 64-bit on AMD Opteron (Linux) and Sun (Solaris) systems.
That is both Sparc Solaris and x86_64 Solaris (although for the latter you seem to need to use the SunStudio compilers). As far as I know all current desktop Intel processors run x86_64, and Xeons seem to have a price-performance edge at the moment. We have several boxes with dual quad-core Xeons and lots of RAM. (Not all for use with R, some Linux, some Windows.) Core 2 Duos do, and are commonplace in quite low-end systems.
2. Does R have any built-in limitations of RAM available to a
session?
For example, could it make use of 16 GB in one session given the
right
processor/OS platform?
R does have built-in limitations even in a 64-bit system, but they are
large. It is certainly possible to use more than 16Gb of memory. The main limit is that the length of a vector is stored in a C int,
and
so is no more than 2^31-1, or about two billion. A numeric vector of that length would take up 16Gb on its own.
?"Memory-limits" documents them.
3. Is there anything else I should consider before choosing a
processor
and OS?
I don't think there is anything else R-specific.
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595