Hi! I ran a GLMM (binomial family) using the package lme4: library(lme4) #GLMM m1 <- glmer(Canto ~ T + H + Patm + V + Pp + CU + (1|Evento), data = Datos, family = binomial) I was wondering if it would be appropriate to evaluate the goodness of fit of the model by applying the Hosmer-Lemeshow test. Thank you very much. Lic. Camila Deutsch Becaria Doctoral-CONICET Grupo de Estudios sobre Biodiversidad en Agroecosistemas Facultad de Ciencias Exactas y Naturales. IEGEBA Universidad de Buenos Aires - CONICET
Hosmer-Lemeshow test in GLMM (binomial family) using lme4 package
4 messages · Camila Deutsch, Ben Bolker, Juho Kristian Ruohonen +1 more
I think it should be appropriate. However, Frank Harrell (a leading biostatistician) https://twitter.com/f2harrell/status/1228423023834718208 "The Hosmer-Lemeshow test has been obsolete for more than a decade. Not recommend. Low power, hard to interpret, very arbitrary to how deciles are computed." You can use https://search.r-project.org/CRAN/refmans/DescTools/html/HosmerLemeshowTest.html (which returns both HL and the recommned le Cessie-van Houwelingen-Copas-Hosmer statistics ...) cheers Ben Bolker
On 2023-07-10 10:27 a.m., Camila Deutsch wrote:
Hi! I ran a GLMM (binomial family) using the package lme4: library(lme4) #GLMM m1 <- glmer(Canto ~ T + H + Patm + V + Pp + CU + (1|Evento), data = Datos, family = binomial) I was wondering if it would be appropriate to evaluate the goodness of fit of the model by applying the Hosmer-Lemeshow test. Thank you very much. Lic. Camila Deutsch Becaria Doctoral-CONICET Grupo de Estudios sobre Biodiversidad en Agroecosistemas Facultad de Ciencias Exactas y Naturales. IEGEBA Universidad de Buenos Aires - CONICET [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
On top of what Frank (and Ben) already mentioned, AFAIK the Hosmer-Lemeshow test is known to overreject GLMMs. The authors of the test acknowledge this themselves (Hosmer, Lemeshow and Sturdivant 2013: 366). Not optimal at all. J Hosmer, David, Stanley Lemeshow and Rodney X Sturdivant. 2013. *Applied Logistic Regression *(3rd ed.). Wiley. ma 10. hein?k. 2023 klo 17.58 Ben Bolker (bbolker at gmail.com) kirjoitti:
I think it should be appropriate. However, Frank Harrell (a leading biostatistician) https://twitter.com/f2harrell/status/1228423023834718208 "The Hosmer-Lemeshow test has been obsolete for more than a decade. Not recommend. Low power, hard to interpret, very arbitrary to how deciles are computed." You can use https://search.r-project.org/CRAN/refmans/DescTools/html/HosmerLemeshowTest.html (which returns both HL and the recommned le Cessie-van Houwelingen-Copas-Hosmer statistics ...) cheers Ben Bolker On 2023-07-10 10:27 a.m., Camila Deutsch wrote:
Hi! I ran a GLMM (binomial family) using the package lme4: library(lme4) #GLMM m1 <- glmer(Canto ~ T + H + Patm + V + Pp + CU + (1|Evento), data =
Datos,
family = binomial) I was wondering if it would be appropriate to evaluate the goodness of
fit
of the model by applying the Hosmer-Lemeshow test.
Thank you very much.
Lic. Camila Deutsch
Becaria Doctoral-CONICET
Grupo de Estudios sobre Biodiversidad en Agroecosistemas
Facultad de Ciencias Exactas y Naturales. IEGEBA
Universidad de Buenos Aires - CONICET
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
2 days later
Ben Bolker skreiv 10.07.2023 16:53:
However, Frank Harrell (a leading biostatistician) https://twitter.com/f2harrell/status/1228423023834718208 "The Hosmer-Lemeshow test has been obsolete for more than a decade.? Not recommend.? Low power, hard to interpret, very arbitrary to how deciles are computed."
Even though the Hosmer?Lemeshow test isn?t very good (for many reasons), the idea behind it, comparing predicted probabilities with relative frequencies, is sound. You do this comparison by plotting *smoothed* observed values against the predicted probabilities using the Hmisc::wtd.loess.noiter() function (compare the results with a abline(0,1) line). This function has some properties that makes it suitable for drawing calibration plots. I?m sure there are other functions you can use for drawing calibration plots, but I like the simplicity of wtd.loess.noiter().
Karl Ove Hufthammer