Skip to content

taking in account results of a gmml in despite of error warning about memory?

4 messages · glenda mendieta, Ben Bolker, Chris Howden +1 more

Original
glenda mendieta
# 
Hi to everyone,

I have being trying to fit aglmm on binay and poisson data and when I 
run this model, with poisson data, the error below shows up. But still 
gives me results. Does this means that those are only partial results 
and shouldn't be taking into account, because the model didn't run fully?
Error: cannot allocate vector of size 183 Kb
In addition:Warning messages:
1: In structure(list(message = as.character(message), call = call),  :
   Reached total allocation of 4061Mb: see help(memory.size)
2: In structure(list(message = as.character(message), call = call),  :
   Reached total allocation of 4061Mb: see help(memory.size)


I am using the latest version of R and R studio. As I have seen before 
that some complicated models don't run at all if I had already many 
other models as objects in the workspace, I did run this one with the 
minimum use of memory (just the database as an object). I also read that 
R is suppose to do not have memory problems any more, but I don't really 
know how to expand the use of memory by R on my pc. For what I observed 
with mem.limits() it appears unlimited (NA), but then if I type 
mem.limit(), 4061 shows up. Does that mean that I can not run those 
models in my pc at al?.
Here, some more info in the data:

Number of obs: 23407, groups:tree,89


Thanks to anyone who can shed some light on this,

Glenda Mendieta-Leiva
PhD candidate
University of Oldenburg
On 28/11/2011 12:00, r-sig-mixed-models-request at r-project.org wrote:
Ben Bolker
# 
glenda mendieta <glendamendieta at ...> writes:
Be very careful.  It probably means that the results are left over
from some previous run that you tried that did work.  An Error
results should *not* give any result.  Try re-running in a clean
R session, with just the minimal stuff you need loaded (see below).
Your random effect specification is not sensible (I think): it asks for an
estimate of the variation of the species effect across trees, which
is more or less impossible because every tree belongs only to a single
species.  Did you mean (1|spp/tree) or (1|spp)+(1|tree) ?
[snip]
Take a look at the R for Windows FAQ entries on memory use.
   Can you give the results of sessionInfo()?  If you can, you
may need to switch to a 64-bit OS.
This does not seem like a particularly huge dataset, so I'm
a bit surprised you're running into trouble (with the exception
of the weird RE specification ...) are your predictor variables
all continuous?
Chris Howden
# 
Hi Glenda,

Try the following with a fresh session of R to maximise memory:
memory.limit(size=4095)
# report memory limit
memory.limit(size=NA)
# maximum amount of memory obtained from the OS is reported
memory.limit(size=TRUE)
# amount currently in use
memory.limit(size=FALSE)


When u clean your workspace using rm(), u may need to run gc() to free up
space.


Also to double check its size, and not something else try running it on a
subset of the data. Although as U have random effects U'll need to insure
the subset has the right data that the random component makes sense.

It may also be due to U estimating a huge covariance matrix, think about
how big it would be.

Chris Howden B.Sc. (Hons) GStat.
Founding Partner
Evidence Based Strategic Development, IP Commercialisation and Innovation,
Data Analysis, Modelling and Training
(mobile) 0410 689 945
(fax) +612 4782 9023
chris at trickysolutions.com.au




Disclaimer: The information in this email and any attachments to it are
confidential and may contain legally privileged information.?If you are
not the named or intended recipient, please delete this communication and
contact us immediately.?Please note you are not authorised to copy, use or
disclose this communication or any attachments without our consent.
Although this email has been checked by anti-virus software, there is a
risk that email messages may be corrupted or infected by viruses or other
interferences. No responsibility is accepted for such interference. Unless
expressly stated, the views of the writer are not those of the company.
Tricky Solutions always does our best to provide accurate forecasts and
analyses based on the data supplied, however it is possible that some
important predictors were not included in the data sent to us. Information
provided by us should not be solely relied upon when making decisions and
clients should use their own judgement.


-----Original Message-----
From: r-sig-mixed-models-bounces at r-project.org
[mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of glenda
mendieta
Sent: Tuesday, 29 November 2011 2:18 AM
To: r-sig-mixed-models at r-project.org
Subject: [R-sig-ME] taking in account results of a gmml in despite of
error warning about memory?

Hi to everyone,

I have being trying to fit aglmm on binay and poisson data and when I
run this model, with poisson data, the error below shows up. But still
gives me results. Does this means that those are only partial results
and shouldn't be taking into account, because the model didn't run fully?
+(0+spp|tree), data=db.e_St, family=poisson(link=log))
Error: cannot allocate vector of size 183 Kb
In addition:Warning messages:
1: In structure(list(message = as.character(message), call = call),  :
   Reached total allocation of 4061Mb: see help(memory.size)
2: In structure(list(message = as.character(message), call = call),  :
   Reached total allocation of 4061Mb: see help(memory.size)


I am using the latest version of R and R studio. As I have seen before
that some complicated models don't run at all if I had already many
other models as objects in the workspace, I did run this one with the
minimum use of memory (just the database as an object). I also read that
R is suppose to do not have memory problems any more, but I don't really
know how to expand the use of memory by R on my pc. For what I observed
with mem.limits() it appears unlimited (NA), but then if I type
mem.limit(), 4061 shows up. Does that mean that I can not run those
models in my pc at al?.
Here, some more info in the data:

Number of obs: 23407, groups:tree,89


Thanks to anyone who can shed some light on this,

Glenda Mendieta-Leiva
PhD candidate
University of Oldenburg
On 28/11/2011 12:00, r-sig-mixed-models-request at r-project.org wrote:
blme.
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
ONKELINX, Thierry
# 
Dear Glenda,

As Chris allready mentioned: if spp has a lot of levels, then you are estimating a huge covariance matrix. Which might be the problem. Do you have different values for spp within a tree? If spp stands for species then that would be rarely the case (except for misclassification). If you have only on level of spp per tree, then you probably want something like (1|spp/tree) which stands for tree nested in spp.

Furthermore you might want to think again on the 3-way interaction term. Do you need it? And if so can you give a biological explanation? If not, I would omit it.

Best regards,

Thierry

ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature and Forest
team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
Kliniekstraat 25
1070 Anderlecht
Belgium
Thierry.Onkelinx at inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey

-----Oorspronkelijk bericht-----
Van: r-sig-mixed-models-bounces at r-project.org [mailto:r-sig-mixed-models-bounces at r-project.org] Namens glenda mendieta
Verzonden: maandag 28 november 2011 16:18
Aan: r-sig-mixed-models at r-project.org
Onderwerp: [R-sig-ME] taking in account results of a gmml in despite of error warning about memory?

Hi to everyone,

I have being trying to fit aglmm on binay and poisson data and when I run this model, with poisson data, the error below shows up. But still gives me results. Does this means that those are only partial results and shouldn't be taking into account, because the model didn't run fully?
Error: cannot allocate vector of size 183 Kb In addition:Warning messages:
1: In structure(list(message = as.character(message), call = call),  :
   Reached total allocation of 4061Mb: see help(memory.size)
2: In structure(list(message = as.character(message), call = call),  :
   Reached total allocation of 4061Mb: see help(memory.size)


I am using the latest version of R and R studio. As I have seen before that some complicated models don't run at all if I had already many other models as objects in the workspace, I did run this one with the minimum use of memory (just the database as an object). I also read that R is suppose to do not have memory problems any more, but I don't really know how to expand the use of memory by R on my pc. For what I observed with mem.limits() it appears unlimited (NA), but then if I type mem.limit(), 4061 shows up. Does that mean that I can not run those models in my pc at al?.
Here, some more info in the data:

Number of obs: 23407, groups:tree,89


Thanks to anyone who can shed some light on this,

Glenda Mendieta-Leiva
PhD candidate
University of Oldenburg
On 28/11/2011 12:00, r-sig-mixed-models-request at r-project.org wrote:
_______________________________________________
R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models