Advice on comparing non-nested random slope models
Hello, This is a bit of a follow-up to a question last week on selecting among GLMM models. Is there a recommended strategy for comparing non-nested, random slope models? I have seen a similar question posted here http://stats.stackexchange.com/questions/116935/comparing-non-nested- models-with-aic but it doesn't seem to answer the problem - and maybe there is no "answer". Zuur et al. (2010) discuss model selection but only in a nested framework. Bolker et al. (2009) suggest AIC can be used in GLMMs but caution against boundary issues and don't specifically mention any issues with comparing different random effects structures (as Zuur does). The context of my question comes from an analysis where we have 5 *a priori* hypotheses describing different climate effects on juvenile recruitment in an ungulate species. The data set has 21 populations (or herds) with repeated annual measurements of recruitment and the climate variables measured at the herd scale. To generate SE's that reflect herd as the sampling unit, explanatory variables are specified as random slopes within herd (as recommended by Schielzeth & Forstmeier 2009; Year is also specified as a random intercept). Because there are only 21 herds, models are fairly simple with only 2-3 explanatory variables (3 may by pushing it...????). I can't post the data but it isn't really relevant to the question (I think). Initially, we looked at AIC to compare models. At the bottom of this email, I have pasted the output from two models, each representing separate hypotheses, to illustrate "the problem". The first model yields an AIC value of 2210.7. The second model yields an AIC of 2479.5. Using AIC, Model 1 would be the "best" model. However, examining the parameter estimates within each model makes me think twice about declaring Model 1 (or the hypothesis it represents) as the most parsimonious explanation for the data. In Model 1, two of the thee fixed effects estimates have small effect sizes and all estimates are "non-significant" (if one considers p-values....). In Model 2, two of the three fixed effect estimates have larger effect sizes are would be considered "significant. Is this an example of the difficulty in using AIC to compare non-nested mixed models.....or am I missing something in my interpretation? I haven't come across this type of result when model selecting among GLMs. Any suggestions on how best to compare competing hypotheses represented by non-nested GLMMs? Should one just compare relative effect sizes of parameter estimates among models? Any help would be appreciated. Thanks, Craig *Model 1:* Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod'] Family: binomial ( logit ) Formula: (Calves/Cows) ~ spr.indvi.ab + green.rate.ab + trend + (1 | Year) + (spr.indvi.ab + green.rate.ab + trend | Herd) Data: bou.dat Weights: Cows *AIC * BIC logLik deviance df.resid *2210.7* 2265.0 -1090.3 2180.7 262 Scaled residuals: Min 1Q Median 3Q Max -3.8700 -1.0800 -0.1057 1.0405 6.8353 Random effects: Groups Name Variance Std.Dev. Corr Year (Intercept) 0.10517 0.3243 Herd (Intercept) 0.29832 0.5462 spr.indvi.ab 0.04331 0.2081 0.38 green.rate.ab 0.03741 0.1934 0.68 0.62 trend 0.62661 0.7916 -0.59 0.20 -0.46 Number of obs: 277, groups: Year, 22; Herd, 21 Fixed effects: Estimate Std. Error z value Pr(>|z|) (Intercept) -1.62160 0.15798 -10.265 <2e-16 *** spr.indvi.ab 0.04019 0.09793 0.410 0.682 green.rate.ab 0.04704 0.05555 0.847 0.397 trend -0.29676 0.23092 -1.285 0.199 --- Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1 Correlation of Fixed Effects: (Intr) spr.n. grn.r. spr.indvi.b -0.113 green.rat.b 0.347 0.438 trend -0.606 0.349 -0.200 *Model 2:* Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod'] Family: binomial ( logit ) Formula: (Calves/Cows) ~ win.bb + tot.sn.ybb + trend + (1 | Year) + (win.bb + tot.sn.ybb | Herd) Data: bou.dat Weights: Cows * AIC* BIC logLik deviance df.resid *2479.5 * 2519.4 -1228.8 2457.5 266 Scaled residuals: Min 1Q Median 3Q Max -4.5720 -1.1801 -0.1364 1.3704 8.3271 Random effects: Groups Name Variance Std.Dev. Corr Year (Intercept) 0.10694 0.3270 Herd (Intercept) 0.13496 0.3674 win.bb 0.05351 0.2313 -0.13 tot.sn.ybb 0.06200 0.2490 0.23 0.34 Number of obs: 277, groups: Year, 22; Herd, 21 Fixed effects: Estimate Std. Error z value Pr(>|z|) (Intercept) -1.851656 0.127702 -14.500 < 2e-16 *** win.bb -0.364019 0.101386 -3.590 0.00033 *** tot.sn.ybb 0.275271 0.118111 2.331 0.01977 * trend -0.007568 0.115706 -0.065 0.94785 --- Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1 Correlation of Fixed Effects: (Intr) win.bb tt.sn. win.bb 0.048 tot.sn.ybb 0.269 0.083 trend -0.242 -0.269 -0.131
Craig DeMars, Ph.D. Postdoctoral Fellow Department of Biological Sciences University of Alberta Phone: 780-221-3971 <(780)%20221-3971> [[alternative HTML version deleted]]