Skip to content

Bug report for lme

8 messages · Douglas Bates, ONKELINX, Thierry, Michal Figurski +3 more

#
On Fri, Mar 11, 2011 at 2:32 PM, Michal Figurski
<figurski at mail.med.upenn.edu> wrote:
... Hundreds of lines deleted
Well, hundreds, perhaps thousands, of lines of (part of)  a dump of an
object are unlikely to be helpful.

To get R working again you probably need to delete the file .RData in
the default directory in which you start R.  It is most likely that
you have a corrupt object in the saved worksheet.

Secondly, using paste to create a formula is not a great idea.  The
formula may have ended up being unevaluated.  If you really do need to
use a character string to name the response variable the preferred
approach is

form <- substitute(foo ~ SPL + (SPL | Plate:Aliq) - 1, list(foo = as.name(i1)))
lmer(form, data=a)

If those suggestions don't help then please provide a reproducible
example and preferrably on less than a gigantic data set.

I am taking the liberty of cc:ing the R-SIG-Mixed-Models at R-project.org
mailing list on this reply.  Many of those who read the list may be
able to help you and it is unlikely I will be able to give much
assistance soon because I am travelling for the next 10 days.
2 days later
#
Douglas,

It seems that I didn't make myself entirely clear, so here are a few 
clarifications. I also attach some data and a reproducible example that 
crashes R -  at least on my machine.

Clarifications:
  - "hundreds of lines" - this was actually just one line in R output. 
This was part of the error message. I selected it and pasted into the 
email body, and it got converted to multiple lines... I agree this is 
unlikely to be helpful, though I was not sure, so I pasted the entire 
error message.
  - formula with 'paste()' is not a problem to me - I use it daily and 
it always works. Though for your convenience, in the attached code I 
used explicit formula - the outcome is unfortunately the same.
  - my .RData file is not corrupt and R starts normally. It only crashes 
after I run the problem code.

The code to reproduce the error:

# ******************************************************************* #
# Working example to reproduce crash
library(lme4)

# Read the data in - replace path with your path
b      <- read.csv("crash-example.csv")
b$Aliq <- factor(b$Aliq)
b$SPL  <- factor(b$SPL)

# 1st example, to show that it works - on a selected set of data
# It takes a few moments, but it works - at least it doesn't crash R:
bb  <- b[b$SPL %in% unique(b$SPL[b$Plate=="TRT1"]),]
b.1 <- lmer(Conc~SPL + (SPL|Plate:Aliq)-1, data=bb)

# 2nd example - it takes only a few seconds and R crashes:
# R-version 2.12.2 on Win7 64bit
# Package lme4 version 0.999375-39
# Previously, when used in R 2.11.1 with lme4 version 0.999375-35,
# I got the error that I pasted in the message before, but the R
# did not crash.
b.2 <- lmer(Conc~SPL + (SPL|Plate:Aliq)-1, data=b)
# ******************************************************************* #

Kind regards,

--
Michal J. Figurski, PhD
HUP, Pathology & Laboratory Medicine
Biomarker Research Laboratory
3400 Spruce St. 7 Maloney S
Philadelphia, PA 19104
tel. (215) 662-3413
On 3/11/2011 4:00 PM, Douglas Bates wrote:
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: crash-example.csv
URL: <https://stat.ethz.ch/pipermail/r-sig-mixed-models/attachments/20110314/3f03ff7c/attachment.pl>
#
Michal,

SPL has 199 levels. Thus (SPL|Plate:Aliq) requires a 199 x 199 variance-covariance matrix, which is huge. 
Have a look at table(b$SPL, factor(b$Plate:b$Aliq)). Notice that you have many combinations without data.
Furthermore you have only 6 levels of Plate:Aliq, which is rather low to get good variance estimates.

Bottomline: your model is to complex for the data you have.

Best regards,

Thierry

----------------------------------------------------------------------------
ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek
team Biometrie & Kwaliteitszorg
Gaverstraat 4
9500 Geraardsbergen
Belgium

Research Institute for Nature and Forest
team Biometrics & Quality Assurance
Gaverstraat 4
9500 Geraardsbergen
Belgium

tel. + 32 54/436 185
Thierry.Onkelinx at inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey
#
Theirry and all,

Thank you for your comments - it is an educative experience for me.

I actually didn't expect the model to work, but I did not expect R to 
crash either. So, here I'm reporting the fact that R crashes.

However, I just noticed that when I remove NAs from the data, R 2.12.2 
survives and displays the same error message as R 2.11.1 did.

--
Michal J. Figurski, PhD
HUP, Pathology & Laboratory Medicine
Biomarker Research Laboratory
3400 Spruce St. 7 Maloney S
Philadelphia, PA 19104
tel. (215) 662-3413
On 3/14/2011 10:53 AM, ONKELINX, Thierry wrote:
#
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 11-03-14 10:53 AM, ONKELINX, Thierry wrote:
So in the short term the answer comes from the old joke: "Doctor, it
hurts when I do this." "Well, then, don't do that."

  In the medium/long term, the standard in R is that it should be
*impossible* to actually crash the R session, no matter how silly the
model one tries to fit.  More ambiguously than that, it is very nice
(though not always possible) to have the code warn you, with as
informative an error message as possible, when it detects that you are
trying to do something silly ...
ir. Thierry Onkelinx
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1+QbMACgkQc5UpGjwzenPiDgCfVHRzFijDxcQMOfYSLzIm6FqP
S0MAn0ETslK1SbpiE9GJEG7f4MZt6LYi
=KeMo
-----END PGP SIGNATURE-----
#
Dear list,

I have a couple of questions related to dotplots and ranef:

1) Does anyone know how to order the distribution of a residuals in a 
dotplot:

dotplot(dotplot(ranef(model, postVar = TRUE)...

sometimes residuals are ordered but other times they are not.

2) Is it possible to organize these plots in a matrix (as par function 
does)? I have more than a random term... so would be useful to order 
theses plots in a matrix...

3) Is it possible to change the strip name of these plots?

Any clue about these questions will be appreciate.
Thank you in  advance!
#
On Mon, Mar 14, 2011 at 12:26 PM, Ben Bolker <bbolker at gmail.com> wrote:
A common problem is that memory is used up yet there is no
mechanism to catch this and abort gracefully. This is not specific
to R. Any application run under Windows or Linux (in my experience)
that grabs huge chunks of the memory available can cripple the OS
and lead to strange error messages from R (perhaps because it cannot
allocate the memory for a meaningful error message!)

I guess there is a working assumption in many programs that the OS
will not run out of memory, and there is no early warning system that
signals when memory resources are running low.

This kind of early warning system should not be difficult to add to
R...

Dominick
6 days later
#
2011/3/14 Sebasti?n Daza <sebastian.daza at gmail.com>:
Can you provide a reproducible example?
We have good news and bad news.  The good news is that you get more
flexibility with lattice graphics than with traditional R graphics
(which is what the par function controls).  The bad news is that it
requires a bit more effort to control the lattice graphics display in
that you need to read the manual page for print.trellis carefully.  An
example of the dotplot for more than one grouping factor is provided
on slide 31 of http://lme4.R-forge.R-project.org/slides/2011-03-16-Amsterdam/1Simple.pdf
 The code that generates that figure is

qrr2 <- dotplot(ranef(fm2, postVar = TRUE), strip = FALSE)
print(qrr2[[1]], pos = c(0,0,1,0.75), more = TRUE)
print(qrr2[[2]], pos = c(0,0.65,1,1))
Yes.  See the description of the strip.default function in the lattice
package.  In particular, the strip can be suppressed by setting
strip=FALSE, as above.