Allocate virtual memory on hard drive

The vector contains 1.5*10^8 numeric elements. It takes about 3~4 GB in memory.
And I would like to find percentiles: 0%, 0.5%, 1%, ... 100%
I use 64 bit R and windows 7 with 24GB Ram.
So:

1) Try R 3.0.0 alpha.  Many operations on large vectors are more 
efficient there.

2) You could try --max-mem-size=32G, say.  In my experience Windows 
virtual memory management is too slow to be useful, but you could try ....

3) Add more RAM.  24GB is not a lot these days.

However, I tried this on a Linux box. Such a vector is only just over 
1GB and the maximum memory usage was 2.9GB.  Have you really told us the 
true story?
Thank you.

Best,

On Mon, Mar 11, 2013 at 12:40 PM, jim holtman <jholtman at gmail.com> wrote:
R runs with data in memory.  What type of system are you running on (32 or
64 bit)?  How big is your data; you did not provide much information about
your problem.  Depending on what you what to 'sort', there might be other
ways of doing it.  This gets back to my tag line: "Tell me what you want to
do, not how you want to do it".

On Mon, Mar 11, 2013 at 11:20 AM, Jie <jimmycloud at gmail.com> wrote:
Dear All,

I have a long sequence and want to find the quantile, or sort it first.
It seems sort() or quantile() reaches the memory limit.
Is there a way to allocate more memoy on SSD for R when startup, so
that R can use both RAM and hard drive space?
Thank you.

Best wishes,
Jie

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

--
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Allocate virtual memory on hard drive

Thread (7 messages)