Making lme4 faster for specific case of sparse x

If X == Z don't you have problems with estimability?  It seems that mle
would always correspond to all random effects being zero.

Perhaps I misunderstand the situation.  Could you provide a bit more
detail on how it comes about that X == Z?

On Mon, Aug 8, 2016 at 5:01 PM Patrick Miller <pmille13 at nd.edu> wrote:

Hello,

For my dissertation, I'm working on extending boosted decision trees to
clustered data.

In one of the approaches I'm considering, I use *lmer* to estimate random
effects within each gradient descent iteration in boosting. As you might
expect, this is computationally intensive. However, my intuition is that
this step could be made faster because my use case is very specific.
Namely, in each iteration, *X = Z*, and *X* is a sparse matrix of 0s and
1s
(with an intercept).

I was wondering if anyone had suggestions or (theoretical) guidance on
this
problem. For instance, is it possible that this special case permits
faster
optimization via specific derivatives? I'm not expecting this to be
implemented in lmer or anything, and I'm happy to work out a basic
implementation myself for this case.

I've read the vignette on speeding up the performance of lmer, and
setting calc.derivs
= FALSE resulted in about a 15% performance improvement for free, which
was
great. I was just wondering if it was possible to go further.

Thanks in advance,

- Patrick

--
Patrick Miller
Ph.D. Candidate, Quantitative Psychology
University of Notre Dame

        [[alternative HTML version deleted]]