Hi, I am not not sure whether this topic belongs to this mail list, but I feel the subscribers here should be the right audience. I noticed that the memory limit reported under Windows is 16 GB. I am wondering how to increase it. I didn't found anything in Rprofile.site nor .Rprofile. Is this limit hard coded at compilation? Best, Samuel
Hard memory limit of 16GB under Windows?
9 messages · Tomas Kalibera, Marc Schwartz, Thierry Onkelinx +3 more
Hi Samuel,
could you please provide more information? Where do you see the limit
reported or how did you trigger it, what version of Windows do you have,
are you using 64-bit build of R (sessionInfo())
Please check help("Memory-limits") and section 8 of R Admin manual
Best,
Tomas
On 4/7/20 1:35 PM, Samuel Granjeaud IR/Inserm wrote:
Hi, I am not not sure whether this topic belongs to this mail list, but I feel the subscribers here should be the right audience. I noticed that the memory limit reported under Windows is 16 GB. I am wondering how to increase it. I didn't found anything in Rprofile.site nor .Rprofile. Is this limit hard coded at compilation? Best, Samuel [[alternative HTML version deleted]]
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Hi Tomas, Many thanks for your answer. Here is a copy of a fresh session under RStudio, and after a copy under Rgui. Strangely enough the result of memory.limit() is not the same. Without your question I would not have looked to RGui, being used to work with RStudio. The value under RGui sounds to correspond to the total RAM of the computer. It makes me noticing that the value is in MB. The value under Rstudio was so huge (1.759219e+13) that I just interpreted it as GB. But I was totally wrong. So in fact I don't know what it refers to. The documentation says "For a 64-bit versions of R under 64-bit Windows the limit is currently 8Tb.", but it looks like being 16TB, which my computer don't have of course. I still have to understand why my colleague has a problem of memory allocation (cannot allocate vector of size 12.6 Gb). Sorry for the wrong interpretation, but thanks for the help, Samuel --- RStudio R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > memory.limit() [1] 1.759219e+13 > sessionInfo() R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C [5] LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3 tools_3.6.3 > --- RGui R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > ls() character(0) > memory.limit() [1] 32627 > sessionInfo() R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 [4] LC_NUMERIC=C LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3 >
Hi Samuel, please also have a look at ?memory.limit. You can set this limit at R startup. It is in megabytes. Maybe R Studio sets it at runtime. Best Tomas
On 4/7/20 3:57 PM, Samuel Granjeaud IR/Inserm wrote:
Hi Tomas, Many thanks for your answer. Here is a copy of a fresh session under RStudio, and after a copy under Rgui. Strangely enough the result of memory.limit() is not the same. Without your question I would not have looked to RGui, being used to work with RStudio. The value under RGui sounds to correspond to the total RAM of the computer. It makes me noticing that the value is in MB. The value under Rstudio was so huge (1.759219e+13) that I just interpreted it as GB. But I was totally wrong. So in fact I don't know what it refers to. The documentation says "For a 64-bit versions of R under 64-bit Windows the limit is currently 8Tb.", but it looks like being 16TB, which my computer don't have of course. I still have to understand why my colleague has a problem of memory allocation (cannot allocate vector of size 12.6 Gb). Sorry for the wrong interpretation, but thanks for the help, Samuel --- RStudio R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
memory.limit()
[1] 1.759219e+13
sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252? LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C [5] LC_TIME=French_France.1252 attached base packages: [1] stats???? graphics? grDevices utils???? datasets? methods base loaded via a namespace (and not attached): [1] compiler_3.6.3 tools_3.6.3
--- RGui R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
ls()
character(0)
memory.limit()
[1] 32627
sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252? LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 [4] LC_NUMERIC=C?????????????????? LC_TIME=French_France.1252 attached base packages: [1] stats???? graphics? grDevices utils???? datasets? methods base loaded via a namespace (and not attached): [1] compiler_3.6.3
Hi Samuel, You may already be aware, but if not, RStudio has their own support mechanisms here: https://support.rstudio.com/hc/en-us If this does turn out to be RStudio specific, you may wish to check there for additional insights. Regards, Marc Schwartz
On Apr 7, 2020, at 10:24 AM, Tomas Kalibera <tomas.kalibera at gmail.com> wrote: Hi Samuel, please also have a look at ?memory.limit. You can set this limit at R startup. It is in megabytes. Maybe R Studio sets it at runtime. Best Tomas On 4/7/20 3:57 PM, Samuel Granjeaud IR/Inserm wrote:
Hi Tomas, Many thanks for your answer. Here is a copy of a fresh session under RStudio, and after a copy under Rgui. Strangely enough the result of memory.limit() is not the same. Without your question I would not have looked to RGui, being used to work with RStudio. The value under RGui sounds to correspond to the total RAM of the computer. It makes me noticing that the value is in MB. The value under Rstudio was so huge (1.759219e+13) that I just interpreted it as GB. But I was totally wrong. So in fact I don't know what it refers to. The documentation says "For a 64-bit versions of R under 64-bit Windows the limit is currently 8Tb.", but it looks like being 16TB, which my computer don't have of course. I still have to understand why my colleague has a problem of memory allocation (cannot allocate vector of size 12.6 Gb). Sorry for the wrong interpretation, but thanks for the help, Samuel --- RStudio R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
memory.limit()
[1] 1.759219e+13
sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C [5] LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3 tools_3.6.3
--- RGui R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
ls()
character(0)
memory.limit()
[1] 32627
sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 [4] LC_NUMERIC=C LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Dear Samuel, The most important information from your mail was the actual error message: "cannot allocate vector of size 12.6 Gb". You'll need to know what code generated this error message. And then figure out if the code does something sensible. Often that is not the case when you get a similar error message. Best regards, ir. Thierry Onkelinx Statisticus / Statistician Vlaamse Overheid / Government of Flanders INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND FOREST Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance thierry.onkelinx at inbo.be Havenlaan 88 bus 73, 1000 Brussel www.inbo.be /////////////////////////////////////////////////////////////////////////////////////////// To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher The plural of anecdote is not data. ~ Roger Brinner The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data. ~ John Tukey /////////////////////////////////////////////////////////////////////////////////////////// <https://www.inbo.be> Op di 7 apr. 2020 om 16:16 schreef Samuel Granjeaud IR/Inserm < samuel.granjeaud at inserm.fr>:
Hi Tomas, Many thanks for your answer. Here is a copy of a fresh session under RStudio, and after a copy under Rgui. Strangely enough the result of memory.limit() is not the same. Without your question I would not have looked to RGui, being used to work with RStudio. The value under RGui sounds to correspond to the total RAM of the computer. It makes me noticing that the value is in MB. The value under Rstudio was so huge (1.759219e+13) that I just interpreted it as GB. But I was totally wrong. So in fact I don't know what it refers to. The documentation says "For a 64-bit versions of R under 64-bit Windows the limit is currently 8Tb.", but it looks like being 16TB, which my computer don't have of course. I still have to understand why my colleague has a problem of memory allocation (cannot allocate vector of size 12.6 Gb). Sorry for the wrong interpretation, but thanks for the help, Samuel --- RStudio R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
> memory.limit()
[1] 1.759219e+13
> sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C [5] LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3 tools_3.6.3
>
--- RGui R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
> ls()
character(0)
> memory.limit()
[1] 32627
> sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 [4] LC_NUMERIC=C LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3
>
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Dear Samuel Does the FAQ for Windows section 2.9 help you here? Michael
On 07/04/2020 12:35, Samuel Granjeaud IR/Inserm wrote:
Hi, I am not not sure whether this topic belongs to this mail list, but I feel the subscribers here should be the right audience. I noticed that the memory limit reported under Windows is 16 GB. I am wondering how to increase it. I didn't found anything in Rprofile.site nor .Rprofile. Is this limit hard coded at compilation? Best, Samuel [[alternative HTML version deleted]]
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Agreed. I don't see any evidence of a "hard limit" here. It just looks like the code you're running has exhausted the memory of your machine. It doesn't require creating a lot of 12.6 Gb vectors to do that, even on a machine with a lot of memory ;-) H.
On 4/7/20 08:05, Thierry Onkelinx via R-devel wrote:
Dear Samuel, The most important information from your mail was the actual error message: "cannot allocate vector of size 12.6 Gb". You'll need to know what code generated this error message. And then figure out if the code does something sensible. Often that is not the case when you get a similar error message. Best regards, ir. Thierry Onkelinx Statisticus / Statistician Vlaamse Overheid / Government of Flanders INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND FOREST Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance thierry.onkelinx at inbo.be Havenlaan 88 bus 73, 1000 Brussel https://urldefense.proofpoint.com/v2/url?u=http-3A__www.inbo.be&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=r6WLJ5dXWo2qb7mQwONaCxYeeWgKwycd3y89JoqY-oY&s=VNmTfuiByvneqd4lcQS0o961TZa3ygo2Rdp7AtiGnk0&e= /////////////////////////////////////////////////////////////////////////////////////////// To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher The plural of anecdote is not data. ~ Roger Brinner The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data. ~ John Tukey /////////////////////////////////////////////////////////////////////////////////////////// <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.inbo.be&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=r6WLJ5dXWo2qb7mQwONaCxYeeWgKwycd3y89JoqY-oY&s=qqsNew3vl9ZvyUMRz_GQy_fAfoj9axzmbcsUUayPVAs&e= > Op di 7 apr. 2020 om 16:16 schreef Samuel Granjeaud IR/Inserm < samuel.granjeaud at inserm.fr>:
Hi Tomas, Many thanks for your answer. Here is a copy of a fresh session under RStudio, and after a copy under Rgui. Strangely enough the result of memory.limit() is not the same. Without your question I would not have looked to RGui, being used to work with RStudio. The value under RGui sounds to correspond to the total RAM of the computer. It makes me noticing that the value is in MB. The value under Rstudio was so huge (1.759219e+13) that I just interpreted it as GB. But I was totally wrong. So in fact I don't know what it refers to. The documentation says "For a 64-bit versions of R under 64-bit Windows the limit is currently 8Tb.", but it looks like being 16TB, which my computer don't have of course. I still have to understand why my colleague has a problem of memory allocation (cannot allocate vector of size 12.6 Gb). Sorry for the wrong interpretation, but thanks for the help, Samuel --- RStudio R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
> memory.limit()
[1] 1.759219e+13
> sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C [5] LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3 tools_3.6.3
>
--- RGui R version 3.6.3 (2020-02-29) -- "Holding the Windsock" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.
> ls()
character(0)
> memory.limit()
[1] 32627
> sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363) Matrix products: default locale: [1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 [4] LC_NUMERIC=C LC_TIME=French_France.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3
>
______________________________________________ R-devel at r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=r6WLJ5dXWo2qb7mQwONaCxYeeWgKwycd3y89JoqY-oY&s=ABvG3sGKR5ln27FVCM8dlmZ82X93ZCTigbMxHeBEb6E&e=
[[alternative HTML version deleted]]
______________________________________________ R-devel at r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=r6WLJ5dXWo2qb7mQwONaCxYeeWgKwycd3y89JoqY-oY&s=ABvG3sGKR5ln27FVCM8dlmZ82X93ZCTigbMxHeBEb6E&e=
Herv? Pag?s Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpages at fredhutch.org Phone: (206) 667-5791 Fax: (206) 667-1319
Thanks to all contributors for your time and useful feedbacks. They pointed me to the right direction and brought me new knowledges. Samuel