Datum: Tue, 20 Dec 2011 09:23:37 -0600
Von: Douglas Bates <bates at stat.wisc.edu>
An: cumuluss at web.de
CC: r-sig-mixed-models at r-project.org
Betreff: Re: [R-sig-ME] problems with allocate memory
On Mon, Dec 19, 2011 at 5:54 PM,? <cumuluss at web.de> wrote:
Hi Douglas,
thanky you for your reply. This is "mydata"
'data.frame':?? 3909896 obs. of? 19 variables:
? $ gr.b??????????? : int? 0 0 0 0 0 0 0 0 0 0 ...
? $ o.ag????? : num? -0.651 -0.651 -0.651 -0.651 -0.651 ...
? $ o.rar????? : num? -0.935 -0.935 -0.935 -0.935 -0.935 ...
? $ si?????? : num? 0.299 0.299 0.299 0.299 0.299 ...
? $ f.ag?????? : num? -1.25 -1.36 -1.33 -1.26 -1.21 ...
? $ f.se?????? : Factor w/ 2 levels "F","M": 1 2 1 2 2 2 1 1 1 1
? $ o.se?????? : Factor w/ 2 levels "F","M": 1 1 1 1 1 1 1 1 1 1
? $ diff??????? : num? -0.536 -0.514 -0.521 -0.534 -0.545 ...
? $ exp.r?????????? : num? -0.168 -0.168 -0.163 -0.168 -0.168
? $ f.rar????? : num? -0.911 0.215 1.224 -1.086 1.107 ...
? $ f.si: num? 1.0008 1.1583 0.0561 -0.4163 0.371 ...
? $ kl????????????? : Factor w/ 3 levels "mat","nonkin",..:
? $ sn????????? : Factor w/ 2 levels "BS","MS": 1 1 1 1 1 1 1 1
? $ MP_y_n????????? : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2
? $ ratio?????? : num? -0.0506 -0.0506 -0.0506 -0.0506 -0.0506
? $ f?????????? : Factor w/ 55 levels "0A0","0A1","0A2",..: 1
6 7 8 9 10 11 13 15 16 ...
? $ o?????????? : Factor w/ 552 levels "","00T","00Z",..: 2 2
? $ d??????????? : int? 9099 9099 9099 9099 9099 9099 9099
? $ MP????????????? : num? 6 6 6 6 5 4 6 6 6 6 ...
formula for the model:
mfit=lmer(gr.b ~ o.ag + o.rar + si + ((f.ag + I(f.ag^2)) * (f.se * (o.se
+ diff + exp.r + f.rar + f.si + kl + sn + MP_y_n + ratio))) +
(1|f)+(1|o)+(1|d) + offset(log(MP)), data=c.data, family=binomial)
I hope this is what you want to see. Thank you for your help.
My guess is that the problem is with creating the fixed-effects model
matrix, of which there could be several copies created during the
evaluation and optimization of the deviance.
Just as a test, could you fit the model for the fixed-effects only
using glm and check on what the size of the model matrix is?
Something like
glm1 <- glm(gr.b ~ o.ag + o.rar + si + ((f.ag + I(f.ag^2)) * (f.se *
(o.se + diff + exp.r + f.rar + f.si + kl + sn + MP_y_n + ratio))) +
offset(log(MP)), data=c.data, family=binomial)
object.size(model.matrix(glm1)
Also, could you convert 'd' to a factor and run str again so we can
learn how many levels there are?? Either that or send the result of
length(unique(mydata$d))
-------- Original-Nachricht --------
Datum: Mon, 19 Dec 2011 14:50:06 -0600
Von: Douglas Bates <bates at stat.wisc.edu>
An: cumuluss at web.de
CC: r-sig-mixed-models at r-project.org
Betreff: Re: [R-sig-ME] problems with allocate memory
On Sun, Dec 18, 2011 at 3:17 PM,? <cumuluss at web.de> wrote:
I have been trying to fit a glmm with a binomial error structure. My
model is a little bit complex. I have 8 continuous predictor variables
them as nonlinear term, 5 categorical predictor variables with some
three-way interactions between them. Additional I have 3 random effects
offset variable in the model. Number of obs is greater than 3million.
I?m working with the latest version of R 2.14.0 on a 64 bit windows
Everything I tried (reducing model complexity, different 64bit PC
even more memory) nothing leads to a fitted model, always the Error
cannot allocate vector of size 2GB.
Is there anything I can do? I would be very grateful for any
You probably have multiple copies of some large objects hanging
around.? Can you send us the output of
str(myData)
where 'myData' is the name of the model frame containing the data you
are using and the formula for the model you are trying to fit?