optimizers for mixed models
Pantelis Hadjipantelis <kalakouentin at ...> writes:
I have personally used uobyqa() from minqa package (not bobyqa()) with success as a fast way to optimize the log-restricted-deviance without using derivatives. I had to move the optimization constraints within the function evaluation (ie. exponentiating everything before being used so I make sure they appear positive, taking cosines to bound parameters with [-1,1]). It appeared rather robust in cases that gradient-assisted BFGS seemed to converge to local minima's.
Good to know. Just to repeat: in many cases the best-fitting model is singular, i.e. located on the constraint boundary of one or more of the theta parameters -- in which case transforming to remove constraints isn't a great idea -- but maybe "the gradient-assisted BFGS seemed to converge to [a] local [minimum]" means precisely that the solution was *not* on this boundary ... this ability to handle singular cases is very convenient, and is in IMO one of lme4's strengths. Ben Bolker
On Sat, Mar 16, 2013 at 7:48 PM, Ben Bolker <bbolker at ...> wrote:
On 3/14/2013 10:55 AM, Ross Boylan wrote:
Optimization was originally via a custom optimizer using rho. The custom optimizer did not incorporate bounds, and blew up when it got rho outside of [-1, 1]. So we switched to atanh(rho) as the target of optimization. However, for some simulated datasets that failed to converge, as atanh(rho) marched slowly off toward infinity. We switched to optim with bounds to cut that process off. So perhaps we should go back to rho, but using optim or the other bounded optimizers you suggested. So the fact that atanh(rho) is unbounded is a feature from some perspectives, but a bug from others.
I forgot to mention that we actually have analytic second (and first) derivatives. Switching to optim from the packages internal optimizer meant we're no longer using the analytic 2nd derivative.
[snip]