An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20121011/929a7cd2/attachment.pl>
Options to extend memory limit
4 messages · jennifer.moeller-gulland at de.pwc.com, Ben Bolker, Sébastien Bihorel +1 more
<jennifer.moeller-gulland <at> de.pwc.com> writes:
at the moment I am using R for calculations of large databases. Unfortunately, R only manages to complete certain operations at some times, and not at others. I usually get the error message "cannot allocate vector of size XX" I am using the 64-bit version with Windows 7. While my computer has 8 RAM, I do have a feeling that R cannot use all of it. Searching online, I found that you can increase the memory with the options --max-mem-size/ --max-ppsize or change the environment variable R_MAX_MEM_SIZE to "allow deep recursion or large and complicated calculations to be done"
I believe this may be somewhat out of date (although I don't use Windows so I'm a little rusty). If you are dealing with large databases you should almost certainly check out the High Performance Computing task view (you can google it), which recommends many approaches for dealing with Big Data.
On Thu, 11 Oct 2012 14:45:16 +0200,
<jennifer.moeller-gulland at de.pwc.com> wrote:
Dear All, at the moment I am using R for calculations of large databases. Unfortunately, R only manages to complete certain operations at some times, and not at others. I usually get the error message "cannot allocate vector of size XX"
I am using the 64-bit version with Windows 7. While my computer has 8 RAM, I do have a feeling that R cannot use all of it.
Searching online, I found that you can increase the memory with the options --max-mem-size/ --max-ppsize or change the environment variable R_MAX_MEM_SIZE to "allow deep recursion or large and complicated calculations to be done"
Unfortunately, I am not very knowledgable yet on how to use R and I did not quite manage to use the commands successfully. Could you please tell me whether these do make sense for my case and if so how (and at what stage of the process) I can use them?
Are you sure you're using the 64 bit R executable which comes with the R installation?
Seb
On Oct 11, 2012, at 9:55 AM, "Sebastian P. Luque" <spluque at gmail.com> wrote:
On Thu, 11 Oct 2012 14:45:16 +0200, <jennifer.moeller-gulland at de.pwc.com> wrote:
Dear All, at the moment I am using R for calculations of large databases. Unfortunately, R only manages to complete certain operations at some times, and not at others. I usually get the error message "cannot allocate vector of size XX"
I am using the 64-bit version with Windows 7. While my computer has 8 RAM, I do have a feeling that R cannot use all of it.
Searching online, I found that you can increase the memory with the options --max-mem-size/ --max-ppsize or change the environment variable R_MAX_MEM_SIZE to "allow deep recursion or large and complicated calculations to be done"
Unfortunately, I am not very knowledgable yet on how to use R and I did not quite manage to use the commands successfully. Could you please tell me whether these do make sense for my case and if so how (and at what stage of the process) I can use them?
Are you sure you're using the 64 bit R executable which comes with the R installation?
Sebastian hit on my initial thought here, though depending upon how much data you are dealing with, 8Gb may indeed not be enough and some of your RAM may be used by other processes/applications, leaving less for R. A quick check to see which version you are running is to use: .Machine$sizeof.pointer If it returns 8, you are using the 64 bit version of R. If it comes back with 4, you are using the 32 bit version of R, which of course will be more limited in how much RAM it can access. If it returns 8, then as Ben noted, you may want to evaluate some of the Large Memory options on the HPC task view: http://cran.r-project.org/web/views/HighPerformanceComputing.html or of course install more RAM. Regards, Marc Schwartz