glmm AIC/LogLik reliability
Hi,
I think AIC is no worse than anything else in this regard, except that it hasn't been explored as carefully as some of the alternatives: thus we suspect by analogy that there are problems similar to those of the LRT, but we don't know for sure. Vaida and Blanchard (2005), Greven (2008), and Burnham and White (2002) are good references. There are
I would also point to the paper by Spiegelhalter et al. (2002) on the DIC. It is a 'Bayesian version' of the DIC but the examples and discussions therein are quite interesting.
two basic issues: (1) if you choose to include models that differ in their random effects components, how do you count "effective" degrees of freedom? (2) how big a sample does it take to reach the "asymptopia" of AIC? If you're not there, what is the best strategy for finite-size correction? If you use AICc, what should you put in for effective residual degrees of freedom?
We are trying to make a comparison of AIC, cAIC (Vaida and Blanchard, 2005) and DIC in this working paper: http://www.bias-project.org.uk/papers/ComparisonSAE.pdf I believe it is a bit of an unfinished work but we have computed several linear (mixed) models in the context of Small Area Estimation and we display the values of AIC/cAIC/DIC in a table for comparison purposes together with the penalty terms. The aim is to study up to what point the AIC, cAIC and DIC are comparable using different structures for the random effects. Any comments are welcome. Hope this helps. Virgilio P.S: Is there any way of obtaining the design matrix of the random effects and the matrix of the variance from an lme object. That would help to compute the cAIC more easily.
Ben Bolker D O S Gillespie wrote:
Dear R-Sig-ME - Lets assume that I am going to use a model averaging AIC based approach to evaluate nested glmm's. I would like to assume that the estimation of AIC and LogLik in the glmm's of lmer are consistent enough (precise, if not accurate) to use in this framework. I realize that we don't trust anova(m1, m2), mainly due to df and tests statistics issues. I realise that some of you may suggest that this is not the correct framework. If so, can you distinguish arguments about the philosophy of AIC model averaging from the practical implementation - i.e. is the output consistent enough to use if, even if you don't believe the answer. Perhaps they are too intertwined. Thanks, Duncan Gillespie
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models