Skip to content

Error messages using LMER

7 messages · Shige Song, Douglas Bates

Original
Shige Song
# 
Dear All,

After playing with lmer for couple of days, I have to say that I am
amazed! I've been using quite some multilevel/mixed modeling packages,
lme4 is a strong candidate for the overall winner, especially for
multilevel generzlized linear models.

Now go back to my two-level poisson model with cross-classified model.
I've been testing various different model specificatios for the past
couple of days. Here are the models I tried:

1) Two level random intercept model with level-1 covariates only
m1 <- lmer(.D ~ offset(log(.Y)) + (1|provn) +(1|bcohort) + x1 + x2 ,
data, poisson, method="Laplace")

2) Two-level random intercept model with both level-1 and level-2
covariates, but no cross-level interactions:
m2 <- lmer(.D ~ offset(log(.Y)) + (1|provn) +(1|bcohort) + x1 + x2 +
z1 + z2, data, poisson, method="Laplace")

3) Two-level random intercept with cross-level interaction
m3 <- lmer(.D ~ offset(log(.Y)) + (1|provn) +(1|bcohort) + x1 + x2 +
z1 + z2 + x1:z1 + x2:z2, data, poisson, method="Laplace")

Both model 1 and 2 run fine. For model 3, I got error message:
----------------------------------
Error in fn(par, ...) : Unable to invert singular factor of downdated X'X
In addition: Warning messages:
1: optim or nlminb returned message ERROR: ABNORMAL_TERMINATION_IN_LNSRCH 
 in: LMEopt(x = mer, value = cv) 
2: Leading minor of size 1 of downdated X'X is indefinite 
----------------------------------

What is going on here? Any workarounds? Thanks!

Best,
Shige
Douglas Bates
#
On 8/18/05, Shige Song <shigesong at gmail.com> wrote:
The first thing I would try is set the EMverbose and msVerbose flags
in the control list to see what occurs within the optimization.  That
is append the argument

control = list(EMverbose = TRUE, msVerbose = TRUE)

to your call to lmer().  You may also want to try the call in a
recently compiled R-devel, which will be released as R-2.2.0 in
October.  You will notice that the first warning message reads "optim
or nlminb". In R-2.1.1 lmer uses optim for the optimization.  Starting
with R-2.2.0 the default is to use nlminb.

Test compilations of R-devel for Windows are available from CRAN.
Shige Song
# 
Dear Professor Bates,

Here is output R 2.1.1 produced with "control = list(EMverbose = TRUE,
msVerbose = TRUE)". I am getting the new devel version and see what
will hapen there:

--------------------------------------------------------------------------------
 EM iterations
 0 85289.766 ( 5407.13:  0.0815) ( 26134.4: 0.00387)
 1 84544.322 ( 333.732:   0.137) ( 1462.32: 0.00934)
 2 84515.108 ( 129.506:  0.0270) ( 446.306: 0.00481)
 3 84514.519 ( 115.592: 0.00355) ( 328.637: 0.00103)
 4 84514.505 ( 113.981:0.000505) ( 311.160:0.000165)
 5 84514.505 ( 113.755:7.45e-005) ( 308.524:2.50e-005)
 6 84514.505 ( 113.722:1.11e-005) ( 308.128:3.77e-006)
 7 84514.505 ( 113.717:1.66e-006) ( 308.068:5.66e-007)
 8 84514.505 ( 113.716:2.50e-007) ( 308.059:8.49e-008)
 9 84514.505 ( 113.716:3.74e-008) ( 308.058:1.27e-008)
 10 84514.505 ( 113.716:5.62e-009) ( 308.058:1.91e-009)
 11 84514.505 ( 113.716:8.43e-010) ( 308.058:2.87e-010)
 12 84514.505 ( 113.716:1.27e-010) ( 308.058:4.31e-011)
 13 84514.505 ( 113.716:1.90e-011) ( 308.057:6.47e-012)
 14 84514.505 ( 113.716:2.86e-012) ( 308.057:9.73e-013)
 15 84514.505 ( 113.716:4.25e-013) ( 308.057:1.44e-013)
iter    0 value 84514.505044
final  value 84514.505044
converged
 EM iterations
 0 83740.342 ( 113.716: -0.0164) ( 308.057:0.000596)
 1 83740.273 ( 121.512:-0.00121) ( 298.914:-3.24e-005)
 2 83740.272 ( 122.131:-0.000111) ( 299.397:-1.24e-005)
iter    0 value 83740.272232
final  value 83740.272232
converged
 EM iterations
 0 84011.550 ( 122.204:-0.00461) ( 299.576:-0.000256)
 1 84011.543 ( 124.624:-0.000459) ( 303.453:-6.41e-005)
 2 84011.543 ( 124.870:-5.42e-005) ( 304.440:-1.13e-005)
iter    0 value 84011.543350
final  value 84011.543350
converged
 EM iterations
 0 84018.592 ( 124.915:-6.44e-005) ( 304.548:-1.22e-005)
 1 84018.592 ( 124.949:-8.29e-006) ( 304.737:-1.99e-006)
 2 84018.592 ( 124.954:-1.15e-006) ( 304.768:-3.08e-007)
iter    0 value 84018.591624
final  value 84018.591624
converged
 EM iterations
 0 84018.612 ( 124.955:3.40e-007) ( 304.770:-1.98e-007)
 1 84018.612 ( 124.955:-9.98e-009) ( 304.773:-2.23e-008)
 2 84018.612 ( 124.955:-5.47e-009) ( 304.774:-2.86e-009)
iter    0 value 84018.611512
final  value 84018.611512
converged
Error in fn(par, ...) : Unable to invert singular factor of downdated X'X
In addition: Warning message:
Leading minor of size 8 of downdated X'X is indefinite
--------------------------------------------------------------------------------

Thanks!

Shige
On 8/18/05, Douglas Bates <dmbates at gmail.com> wrote:
Shige Song
# 
Here is what happened using R-devel:

-------------------------------------------------------------
  EM iterations
  0 85289.766 ( 5407.13:  0.0815) ( 26134.4: 0.00387)
  1 84544.322 ( 333.732:   0.137) ( 1462.32: 0.00934)
  2 84515.108 ( 129.506:  0.0270) ( 446.306: 0.00481)
  3 84514.519 ( 115.592: 0.00355) ( 328.637: 0.00103)
  4 84514.505 ( 113.981:0.000505) ( 311.160:0.000165)
  5 84514.505 ( 113.755:7.45e-005) ( 308.524:2.50e-005)
  6 84514.505 ( 113.722:1.11e-005) ( 308.128:3.77e-006)
  7 84514.505 ( 113.717:1.66e-006) ( 308.068:5.66e-007)
  8 84514.505 ( 113.716:2.50e-007) ( 308.059:8.49e-008)
  9 84514.505 ( 113.716:3.74e-008) ( 308.058:1.27e-008)
 10 84514.505 ( 113.716:5.62e-009) ( 308.058:1.91e-009)
 11 84514.505 ( 113.716:8.43e-010) ( 308.058:2.87e-010)
 12 84514.505 ( 113.716:1.27e-010) ( 308.058:4.31e-011)
 13 84514.505 ( 113.716:1.90e-011) ( 308.057:6.47e-012)
 14 84514.505 ( 113.716:2.86e-012) ( 308.057:9.73e-013)
 15 84514.505 ( 113.716:4.25e-013) ( 308.057:1.44e-013)
  EM iterations
  0 83740.342 ( 113.716: -0.0164) ( 308.057:0.000596)
  1 83740.273 ( 121.512:-0.00121) ( 298.914:-3.24e-005)
  2 83740.272 ( 122.131:-0.000111) ( 299.397:-1.24e-005)
  EM iterations
  0 84011.546 ( 122.131:-0.00474) ( 299.397:-0.000265)
  1 84011.539 ( 124.616:-0.000472) ( 303.415:-6.62e-005)
  2 84011.539 ( 124.869:-5.58e-005) ( 304.433:-1.16e-005)
  EM iterations
  0 84018.589 ( 124.869:-0.000139) ( 304.433:-1.81e-005)
  1 84018.589 ( 124.944:-1.62e-005) ( 304.713:-3.26e-006)
  2 84018.589 ( 124.953:-2.12e-006) ( 304.764:-5.23e-007)
  EM iterations
  0 84018.611 ( 124.953:-2.38e-006) ( 304.764:-5.44e-007)
  1 84018.611 ( 124.954:-3.25e-007) ( 304.772:-8.50e-008)
  2 84018.611 ( 124.955:-4.66e-008) ( 304.773:-1.29e-008)
  EM iterations
  0 84018.611 ( 124.955:-4.75e-008) ( 304.773:-1.30e-008)
  1 84018.611 ( 124.955:-6.93e-009) ( 304.774:-1.97e-009)
  2 84018.611 ( 124.955:-1.03e-009) ( 304.774:-2.96e-010)
Warning messages:
1: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
2: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
3: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
4: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
5: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
6: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
7: nlminb failed to converge in: lmer(.D ~ offset(log(.Y)) + (1 |
provn) + (1 | bcohort) + agri +
-------------------------------------------------------------

Shige
On 8/19/05, Shige Song <shigesong at gmail.com> wrote:
Shige Song
# 
One thing to be noted: after switching to R-devel, even the simplest
model can not converge. I always get this:

-------------------------------------------------------------------------
Warning messages:
1: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
2: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
3: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
4: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
5: optim or nlminb returned message See PORT documentation.  Code (27) 
 in: LMEopt(x = mer, value = cv) 
6: nlminb failed to converge in: lmer(.D ~ offset(log(.Y)) + (1 |
provn) + (1 | bcohort) + educy +
-------------------------------------------------------------------------

The same model did not have problems converging in R 2.1.1.

Shige
On 8/19/05, Shige Song <shigesong at gmail.com> wrote:
Douglas Bates
# 
Thanks for including all of that output.  

I believe that in this version the parameters are the relative
variances.  This would indicate that somehow you are getting fits with
very low residual sums of squares in the weighted least squares
problem.  It could be that you have too many fixed effects terms in
the model and are getting complete separation.
On 8/18/05, Shige Song <shigesong at gmail.com> wrote:
2 days later
Douglas Bates
# 
Several people have been helping examine the cause of the warning
message about Code(27).  The latest message from Duncan Murdoch said
that he encounters the warning when using an older compiler but now
when using the latest compiler.  I hope this means that a new version
of R-devel compiled for Windows does not show this behavior.
On 8/18/05, Douglas Bates <dmbates at gmail.com> wrote: