Hi, Firstly, I'm sorry if this question has been asked (and answered) before (although I couldn't find it). I'm doing my analysis using logit mixed models (family = binomial(link = "logit"), glmer model) and I was wondering if there was a way to calculate power for my final model. I have looked over the web and all I can find are programs and simulations for linear mixed models with a continuous outcome, but not if the outcome is only 0's and 1's. Is there a way for me to calculate power for this model? Thank you very much, Evelien
Logit mixed model power analysis
6 messages · Evelien Heyselaar, paul debes, Thierry Onkelinx +3 more
Hi Evelien & List, A very recent publication might answer your question: http://onlinelibrary.wiley.com/doi/10.1111/2041-210X.12306/pdf Best, Paul On Mon, 14 Sep 2015 10:58:57 +0300, Evelien Heyselaar <ev.heys at gmail.com> wrote:
Hi, Firstly, I'm sorry if this question has been asked (and answered) before (although I couldn't find it). I'm doing my analysis using logit mixed models (family = binomial(link = "logit"), glmer model) and I was wondering if there was a way to calculate power for my final model. I have looked over the web and all I can find are programs and simulations for linear mixed models with a continuous outcome, but not if the outcome is only 0's and 1's. Is there a way for me to calculate power for this model? Thank you very much, Evelien [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Paul Debes DFG Research Fellow University of Turku Department of Biology It?inen Pitk?katu 4 20520 Turku Finland
1 day later
Dear Evenlien, Post-hoc power tests are not very informative. You will get a high power when the signal is significant and low power when not significant. You can always use a brute force approach to estimate the power. Simulate a dataset with know effect size. Analyse that dataset with your model and store the relevant p-values. Repeat this so you get N simulated datasets for that specific effect size. Then power = mean(p < alpha). Best regards, ir. Thierry Onkelinx Instituut voor natuur- en bosonderzoek / Research Institute for Nature and Forest team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance Kliniekstraat 25 1070 Anderlecht Belgium To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher The plural of anecdote is not data. ~ Roger Brinner The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data. ~ John Tukey 2015-09-14 9:58 GMT+02:00 Evelien Heyselaar <ev.heys at gmail.com>:
Hi,
Firstly, I'm sorry if this question has been asked (and answered) before
(although I couldn't find it). I'm doing my analysis using logit mixed
models (family = binomial(link = "logit"), glmer model) and I was wondering
if there was a way to calculate power for my final model. I have looked
over the web and all I can find are programs and simulations for linear
mixed models with a continuous outcome, but not if the outcome is only 0's
and 1's. Is there a way for me to calculate power for this model?
Thank you very much,
Evelien
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Hi Evenlien, I second what Thierry said about the pointlessness of post-hoc power analysis when using the observed effect size. This tutorial by Ben Bolker shows how to do power analysis for a binomial GLMM: http://rpubs.com/bbolker/11703 NB you don?t need the first 3 lines any more (assuming you have a reasonably up-to-date version of lme4). I also recommend Chapter 5 of Ben?s book Ecological Models and Data in R as an introduction to simulation, including power analysis. Our paper on simulation-based power analysis for GLMMs http://dx.doi.org/10.1111/2041-210X.12306 which Paul Debes mentioned, has a supplementary online R tutorial, including a binomial example. Best wishes, Paul
On 15 Sep 2015, at 16:32, Thierry Onkelinx <thierry.onkelinx at inbo.be> wrote: Dear Evenlien, Post-hoc power tests are not very informative. You will get a high power when the signal is significant and low power when not significant. You can always use a brute force approach to estimate the power. Simulate a dataset with know effect size. Analyse that dataset with your model and store the relevant p-values. Repeat this so you get N simulated datasets for that specific effect size. Then power = mean(p < alpha). Best regards, ir. Thierry Onkelinx Instituut voor natuur- en bosonderzoek / Research Institute for Nature and Forest team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance Kliniekstraat 25 1070 Anderlecht Belgium To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher The plural of anecdote is not data. ~ Roger Brinner The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data. ~ John Tukey 2015-09-14 9:58 GMT+02:00 Evelien Heyselaar <ev.heys at gmail.com>:
Hi,
Firstly, I'm sorry if this question has been asked (and answered) before
(although I couldn't find it). I'm doing my analysis using logit mixed
models (family = binomial(link = "logit"), glmer model) and I was wondering
if there was a way to calculate power for my final model. I have looked
over the web and all I can find are programs and simulations for linear
mixed models with a continuous outcome, but not if the outcome is only 0's
and 1's. Is there a way for me to calculate power for this model?
Thank you very much,
Evelien
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
On Wed, 16 Sep 2015, Paul Johnson wrote:
I second what Thierry said about the pointlessness of post-hoc power analysis when using the observed effect size.
Post-hoc power tests are not very informative. You will get a high power when the signal is significant and low power when not significant.
A few people defend it as an estimate of the reproducibility probability. Gelman and Carlin suggest plugging in a "sensible" (Bayesian) estimate of the true effect based on your prior knowledge, so you can estimate the probability of a "Type S" (sign of true effect could actually be opposite to your study point estimate) or "Type M" error. Cheers, David. | David Duffy (MBBS PhD) | email: David.Duffy at qimrberghofer.edu.au ph: INT+61+7+3362-0217 fax: -0101 | Genetic Epidemiology, QIMR Berghofer Institute of Medical Research | 300 Herston Rd, Brisbane, Queensland 4006, Australia GPG 4D0B994A
I understand post-hoc power as being a function of the p-value, and therefore redundant. When asked for the kind of information that post-hoc power is used to represent, I prefer to provide confidence intervals on the parameters of interest - if possible. Best wishes, Andrew On Wed, Sep 16, 2015 at 11:29 AM, David Duffy <
David.Duffy at qimrberghofer.edu.au> wrote:
On Wed, 16 Sep 2015, Paul Johnson wrote: I second what Thierry said about the pointlessness of post-hoc power
analysis when using the observed effect size.
Post-hoc power tests are not very informative. You will get a high power when the signal is significant and low power when not significant.
A few people defend it as an estimate of the reproducibility probability. Gelman and Carlin suggest plugging in a "sensible" (Bayesian) estimate of the true effect based on your prior knowledge, so you can estimate the probability of a "Type S" (sign of true effect could actually be opposite to your study point estimate) or "Type M" error. Cheers, David. | David Duffy (MBBS PhD) | email: David.Duffy at qimrberghofer.edu.au ph: INT+61+7+3362-0217 fax: -0101 | Genetic Epidemiology, QIMR Berghofer Institute of Medical Research | 300 Herston Rd, Brisbane, Queensland 4006, Australia GPG 4D0B994A
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Andrew Robinson Deputy Director, CEBRA, School of Biosciences Reader & Associate Professor in Applied Statistics Tel: (+61) 0403 138 955 School of Mathematics and Statistics Fax: +61-3-8344 4599 University of Melbourne, VIC 3010 Australia Email: a.robinson at ms.unimelb.edu.au Website: http://www.ms.unimelb.edu.au/~andrewpr MSME: http://www.crcpress.com/product/isbn/9781439858028 FAwR: http://www.ms.unimelb.edu.au/~andrewpr/FAwR/ SPuR: http://www.ms.unimelb.edu.au/spuRs/ [[alternative HTML version deleted]]