You can have a look at proper scoring rules: https://en.m.wikipedia.org/wiki/Scoring_rule
These are, for example, calculated by the scoring_rules() function in the GLMMadaptive package (https://drizopoulos.github.io/GLMMadaptive/); for an example check here: https://drizopoulos.github.io/GLMMadaptive/articles/Dynamic_Predictions.html
Best,
Dimitris
From: Williamson, Michael via R-sig-mixed-models <r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>>
Date: Tuesday, 21 May 2019, 17:35
To: r-sig-mixed-models at r-project.org <r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>>
Subject: [R-sig-ME] predict function for a glmm
Good Afternoon,
I am running a GLMM model using the glmmTMB function. I've been told I should run the model on 80% of my data, then test the outputs on the remaining 20% using the predict function in order to test the robustness of the model. I have a binomial response variable (off shore = 1 not offshore =0) for the model.
My code is this:
OffMod_80 <- glmmTMB(offshore ~ sex + log(size) + species*daynight+ species*season +
(1|code), family=binomial(), data=Off_80)
pred_Off20 <- as.data.frame(predict(OffMod_80, newdata = Off_20))
This spits out a table of values from the predict function.
My question is, considering my response variable is non-normal what method can I use to compare these predicted outputs with my observed values to test the robustness of the model? Is this even the correct way to go about things for this data and model type?
Any help appreciated.
Michael Williamson
London NERC DTP Candidate
Email: michael.williamson at kcl.ac.uk<mailto:michael.williamson at kcl.ac.uk> Phone: +447764836592 Skype: mikejwilliamson Twitter: @mjw_marine
Most recent paper:
Williamson, M. J., Tebbs, E., Dawson, T., Jacoby D. (2019) 'Satellite Remote Sensing in Shark and Ray Ecology, Conservation and Management', Frontiers in Marine Science, 6, 1-23. https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.3389%2Ffmars.2019.00135&data=02%7C01%7Cd.rizopoulos%40erasmusmc.nl%7C6efc655e1c334ec70e4308d6de01ea6c%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C636940497136772882&sdata=xCAKq0gw%2BJVtvtWwIZX5WLPr2S3s%2FBrBin%2FDSz62rEk%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.3389%2Ffmars.2019.00135&data=02%7C01%7Cd.rizopoulos%40erasmusmc.nl%7C6efc655e1c334ec70e4308d6de01ea6c%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C636940497136772882&sdata=xCAKq0gw%2BJVtvtWwIZX5WLPr2S3s%2FBrBin%2FDSz62rEk%3D&reserved=0>
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models&data=02%7C01%7Cd.rizopoulos%40erasmusmc.nl%7C6efc655e1c334ec70e4308d6de01ea6c%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C636940497136772882&sdata=JhSqa2R0rlRZ3e6KXFvoRAW8fg39tOUXWlsSyzrxmjg%3D&reserved=0