CLMM: Calculate ICC & Assessing Model Fit

Just voicing two thoughts as a non-statistician:

1. Why do you feel you need a mixed model in the first place? It looks to me like every Individual is contributing the same number of responses (i.e. 4). Thus, the individual-specific effects can be assumed to cancel each other out, and no random effect is needed.
2. Why complicate things with a non-canonical link function? It looks to me like a standard ordinal logistic regression model, with fixed effects only, would do the job for you. Guides and textbooks abound on how to assess the fit of such models.

Best,

J

ke 19. elok. 2020 klo 14.44 Sidoti, Salvatore A. (sidoti.23 at buckeyemail.osu.edu<mailto:sidoti.23 at buckeyemail.osu.edu>) kirjoitti:
To begin with, I'm not a fan of cross-posting. However, I posted my question on Stack Exchange more than two weeks ago, but I have yet to receive a sufficient answer:

https://stats.stackexchange.com/questions/479600/data-with-ordinal-responses-calculate-icc-assessing-model-fit<https://urldefense.com/v3/__https://stats.stackexchange.com/questions/479600/data-with-ordinal-responses-calculate-icc-assessing-model-fit__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6iZNEz0M$>

Here's what I've learned since then (hopefully):

1) ICC of a CLMM:
Computed like this:
(variance of the random effect) / (variance of the random effect + 1) If this is correct, I would love to see a reference/citation for it.

2) 95% Confidence Interval for the ICC from a CLMM Model To my current understanding, a confidence interval for an ICC is only obtainable via simulation. I've conducted simulations with GLMM model objects ('lme4' package) and the bootMer() function. Unfortunately, bootMer() will not accept a CLMM model ('ordinal' package).

3) Model Fit of a CLMM
Assuming that the model converges without incident, the model summary includes a condition number of the Hessian ('cond.H'). This value should be below 10^4 for a "good fit". This is straightforward enough. However, I am not as sure about the value for 'max.grad', which needs to be "well below 1". The question is, to what magnitude should max.grad < 1 for a decent model fit? My reference is linked below (Christensen, 2019), but it does not elaborate further on this point:

https://documentcloud.adobe.com/link/track?uri=urn:aaid:scds:US:b6a61fe2-b851-49ce-b8b1-cd760d290636<https://urldefense.com/v3/__https://documentcloud.adobe.com/link/track?uri=urn:aaid:scds:US:b6a61fe2-b851-49ce-b8b1-cd760d290636__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR65za2Jag$>

3) Effect Size of a CLMM
The random variable's effect is determined by a comparison between the full model to a model with only the fixed effects via the anova() function. I found this information on the 'rcompanion' package website:

https://rcompanion.org/handbook/G_12.html<https://urldefense.com/v3/__https://rcompanion.org/handbook/G_12.html__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6uPeTzGc$>

The output of this particular anova() will include a value named 'LR.stat', the likelihood ratio statistic. The LR.stat is twice the difference of each log-likelihood (absolute value) of the respective models. Is LR.stat the mixed-model version of an "effect size"? If so, how does one determine if the effect is small, large, in-between, etc?

Cheers,
Sal

Salvatore A. Sidoti
PhD Candidate
Behavioral Ecology
The Ohio State University

_______________________________________________
R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models<https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6KhTBL3M$>