To begin with, I'm not a fan of cross-posting. However, I posted my
question on Stack Exchange more than two weeks ago, but I have yet to
receive a sufficient answer:
https://stats.stackexchange.com/questions/479600/data-with-ordinal-responses-calculate-icc-assessing-model-fit
Here's what I've learned since then (hopefully):
1) ICC of a CLMM:
Computed like this:
(variance of the random effect) / (variance of the random effect + 1) If
this is correct, I would love to see a reference/citation for it.
2) 95% Confidence Interval for the ICC from a CLMM Model To my current
understanding, a confidence interval for an ICC is only obtainable via
simulation. I've conducted simulations with GLMM model objects ('lme4'
package) and the bootMer() function. Unfortunately, bootMer() will not
accept a CLMM model ('ordinal' package).
3) Model Fit of a CLMM
Assuming that the model converges without incident, the model summary
includes a condition number of the Hessian ('cond.H'). This value should be
below 10^4 for a "good fit". This is straightforward enough. However, I am
not as sure about the value for 'max.grad', which needs to be "well below
1". The question is, to what magnitude should max.grad < 1 for a decent
model fit? My reference is linked below (Christensen, 2019), but it does
not elaborate further on this point:
https://documentcloud.adobe.com/link/track?uri=urn:aaid:scds:US:b6a61fe2-b851-49ce-b8b1-cd760d290636
3) Effect Size of a CLMM
The random variable's effect is determined by a comparison between the
full model to a model with only the fixed effects via the anova() function.
I found this information on the 'rcompanion' package website:
https://rcompanion.org/handbook/G_12.html
The output of this particular anova() will include a value named
'LR.stat', the likelihood ratio statistic. The LR.stat is twice the
difference of each log-likelihood (absolute value) of the respective
models. Is LR.stat the mixed-model version of an "effect size"? If so, how
does one determine if the effect is small, large, in-between, etc?
Cheers,
Sal
Salvatore A. Sidoti
PhD Candidate
Behavioral Ecology
The Ohio State University