lmer fails when too many observations

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

  Just a quick update on this.
  This 'failure' is coming because (1) the current version of lme4 has
too strict a test for convergence and gives what we think are
false-positive warnings, especially for large data sets; (2) you have
set options(warn=2) so that warnings get converted into errors.  Until
we get the issue fixed (which could take a while, as we haven't yet
given up on finding a more principled way than just increasing the
default tolerance a lot) you can either (a) convert your warnings back
to regular warnings (i.e. options(warn=1) or options(warn=0) or (b)
increase the tolerance level, e.g.

 control= lmerControl(check.conv.grad = .makeCC("warning", tol = 1e-2))

or

 control = lmerControl(check.conv.grad = "ignore")



On 15-03-09 09:52 PM, Asaf Weinstein wrote:

Dear lmer community,

I am trying to run a simulation for a two-way random-effects model
with unbalanced design (ie, unequal number of observations per
cell) and no interaction. It's especially important for me to be
able to run the lmer/blmer functions when the number of (column and
row) random effects is large, say 100, and with possible replicates
in each cell. The problem is that lmer() works with the full vector
of observations, as opposed to working with the cell averages
(which is a sufficient statistic), and the methods fails pretty
quickly when there are replicates (because the response vector is
too big, I suppose). I get the following error:

*Error in get("checkConv", lme4Env)(attr(opt, "derivs"), opt$par,
ctrl = control$checkConv,  : * *  (converted from warning) Model
failed to converge with max|grad| = 0.00244385 (tol = 0.002)*

Just to give an example: suppose there are R=100 row effects, C=100
column effects, and 5 replicates in each cell. The vector of
individual observations is of length 100^5 (lmer fails), while the
vector of cell averages is of length 100^2 (a size which causes no
problem for lmer). My question is whether there is a way to tell
lmer() to work with the sufficient statistic (of course, the
conditional covariance is no longer c*Identity, a fact which is
used in the implementation of lmer (according to documentation) ).

Thank you very much and I hope I was clear!

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJVALMaAAoJEOCV5YRblxUHnRwH/j8/MFNfYKgvgzbBBCdmfXOi
nfRGRpGWrFUM7MznFWFf4hdA0SFPUOTS9yZYxoGuyJHos1q5WeAyR9kdlqNQjdDx
lPV+zYy9gsj+uMl5XBVSurDrfJY+JXfnqtOzZECMbrZjNxHgTMnSxzMdRNVZAVm6
DaCAaNGJkeTI96k4jg7ucj1p/xGgk3deVW8NMY3bKMasd+dtFzTXqPDvEYovBRQT
sbQ7SavzjJY93QYHabaaDkpsD8N86qe5fJgOE9uVLNkvuOGVPELvwG8lCaC/kj7y
wdd4+HOum1AgeBUosCRfs+Ld41V4wBUySxFW9coLRUVwzQGfU8rOTjIpKs4iCpY=
=jitR
-----END PGP SIGNATURE-----