Skip to content

lme4 error: "Number of levels of a grouping factor . . ."

6 messages · Gabrielle Miller-Messner, Martin Maechler, Ben Bolker +1 more

Original
Martin Maechler
# 
> Hello, I have count/proportion data and I am trying to
    > account for overdispersion in a logistic regression. With
    > the following code from lme4 I receive the following error
    > message:

    >> glmm = glmer(cbind(fertilized, unfert) ~ density.class +
    >                (1|individual), family = binomial) 
    > Number of levels of a grouping factor for the random effects
    > is *equal* to n, the number of > observations

    > This problem was mentioned in Warton and Hui 2011, so I
    > imagine it may have been fixed recently.  I downloaded the
    > most recent version of lme4 available through the command
    > install.packages("lme4").  Is there an updated version of
    > the package available in a different form?

{yes, lme4a on R-forge, but that's not the issue now, and not
 something I'd recommend to get into just right now ...}

Well, I'm puzzled that you say you get an *error* with the above
message.
You should get a message, but no error in the case of glmer().

Here's a data generating utility and an example with "poisson"
(from the package's  lme4/tests/lmer-1.R !)
where we show *that* indeed, it's possible and sometimes
sensible to use the "1 random effect per observation" trick
in order to "get around" overdispersion:

## glmer - Modeling overdispersion as "mixture" aka
## ----- - *ONE* random effect *PER OBSERVATION" -- example inspired by Ben Bolker:

##' <description>
##'
##' <details>
##' @title
##' @param ng number of groups
##' @param nr number of "runs", i.e., observations per groups
##' @param sd standard deviations of group and "Individual" random effects,
##'    (\sigma_f, \sigma_I)
##' @param b  true beta (fixed effects)
##' @return a data frame (to be used in glmer()) with columns
##'    (x, f, obs, eta0, eta, mu, y), where y ~ Pois(lambda(x)),
##'                                   log(lambda(x_i)) = b_1 + b_2 * x + G_{f(i)} + I_i
##'    and G_k ~ N(0, \sigma_f);  I_i ~ N(0, \sigma_I)
##' @author Ben Bolker and Martin Maechler
rPoisGLMMi <- function(ng, nr, sd=c(f = 1, ind = 0.5), b=c(1,2))
{
  stopifnot(nr >= 1, ng >= 1,
            is.numeric(sd), names(sd) %in% c("f","ind"), sd >= 0)
  ntot <- nr*ng
  b.reff <- rnorm(ng,  sd= sd[["f"]])
  b.rind <- rnorm(ntot,sd= sd[["ind"]])
  x <- runif(ntot)
  within(data.frame(x,
                    f = factor(rep(LETTERS[1:ng], each=nr)),
                    obs = 1:ntot,
                    eta0 = cbind(1, x) %*% b),
     {
         eta <- eta0 + b.reff[f] + b.rind[obs]
         mu <- exp(eta)
         y <- rpois(ntot, lambda=mu)
     })
}
dd <- rPoisGLMMi(12, 20)
m0  <- glmer(y~x + (1|f),           family="poisson", data=dd)
(m1 <- glmer(y~x + (1|f) + (1|obs), family="poisson", data=dd))
anova(m0, m1)

----------

Could you produce a rBinGLMMi() data generating function from
the above example and use it for  family = "binomial" ,
hence using a reproducible example...

I'm really startled that you say you get an error,
as the change to allow this was {according to the ChangeLog}:

  --------------------------------------------------------------
  2010-06-03  Martin Maechler  <maechler at stat.math.ethz.ch>

	  * DESCRIPTION (Date, Version): 0.999375-34

	  * R/lmer.R (glmer_finalize): for a glmer(), allow q == n

	  * tests/lmer-1.R (rPoisGLMMi): test example, originating from Ben Bolker
  --------------------------------------------------------------

i.e. more than a year ago..

Just to be sure:  What does  
     packageVersion("lme4")
say for you?

Martin


    > I am using R version 2.13.1 on OS 10.6.4.

    > thank you!  Gabrielle

    > -- 
    > Gabrielle Miller-Messner Graduate Student Center for
    > Population Biology Section of Evolution and Ecology
    > University of California, Davis 1 Shields Avenue Davis,
    > California 95616 gmessner at ucdavis.edu

    > 	[[alternative HTML version deleted]]

    > _______________________________________________
    > R-sig-mixed-models at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Ben Bolker
# 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 07/27/2011 03:29 AM, Martin Maechler wrote:
Probably this is indeed just a warning message and not an error?
   Gabrielle, does the function return an answer (i.e. a fitted model)?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4wE90ACgkQc5UpGjwzenPiQACeJ33UCdFDFRUi3V3wh4VHEk+G
KFMAnjA3LOzKK4pj7VnqbzmYaukQT6FV
=jGz9
-----END PGP SIGNATURE-----
Brent Pedersen
#
On Wed, Jul 27, 2011 at 7:34 AM, Ben Bolker <bbolker at gmail.com> wrote:
Hi, I've hit this as well. with:
[1] ?0.999375.67?

(should I be using another version?)
and code:

"""
library(lme4a)

d = data.frame(a=runif(200),
               b=sample(c(strsplit("ABCDEF", "")[[1]]), 200, replace=T),
               c=as.factor(1:200))
# d$c[2] = d$c[1]

fm = glmer(a ~ b + (1|c), d)
"""
it shows:
"""
Error in lmer(formula = a ~ b + (1 | c), data = d) :
  number of levels of each grouping factor must be less than number of obs
"""

and if i uncomment that line to make the number of grouping factors (c) less
than the number of obs, it runs successfully.
not sure if this is different for binomial...
Ben Bolker
# 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 07/27/2011 01:50 PM, Brent Pedersen wrote:
This is different because (despite using the function "glmer" and not
"lmer"), you are actually fitting a linear mixed model (LMM) and not a
generalized linear mixed model (GLMM) here.  (If you don't specify a
family, or you specify a gaussian family with an identity link, glmer
automatically reverts to calling lmer -- this is a design decision
(infelicity?) that I hope we will get around to reversing ...)  In the
case of a LMM, it doesn't make sense to fit a separate individual-level
random factor, because it is completely confounded with the residual
error.  You only need to worry about overdispersion, and hence it only
makes sense to fit individual-level random effects, in cases where the
scale parameter is fixed (e.g. binomial, Poisson).

   Ben Bolker
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4wVJwACgkQc5UpGjwzenOe5gCfRy8EeieSVwXSxbqnVKTS1E79
VeIAn3z4QqAvAD1lxYCFHrzFWlHX6qkH
=Ckg5
-----END PGP SIGNATURE-----
Brent Pedersen
#
On Wed, Jul 27, 2011 at 12:10 PM, Ben Bolker <bbolker at gmail.com> wrote:
that makes sense.
thanks.