Hello everybody, I am trying to apply the outlier and influence diagnostics as described by Viechtbauer and Cheung (2010) on a meta-analysis I conducted, using a four-level random-effects model with 528 included effect sizes. I have calculated the Cook's Distances and DFBETAS in R, using the functions for model diagnostics for rma.mv objects as described here: https://wviechtb.github.io/metafor/reference/influence.rma.mv.html However, I don't know how to decide about which studies are influential based on these values. I read the paper by Viechtbauer and Cheung, but I don't quite understand it. Is there some kind of formula to determine the threshold values of CD and DFBETAS for a meta-analytic model with k outcomes? Thanks in advance for your help. Best, Theresa
[R-meta] Threshold values for Cook's Distances and DFBETAS
4 messages · Thölking, Theresa, Michael Dewey, Wolfgang Viechtbauer
Dear Theresa My feeling is that these diagnostics are better seen as identifying values which may benefit from further investigation. If I was concerned that they were unduly influential in the model I might try fitting leave-one-out models but since those models are essentially data-driven they are also only useful in an exploratory sense. So, even if there is a formula I would be sceptical about its value in answering the scientific question. Michael
On 29/09/2022 21:35, Th?lking, Theresa wrote:
Hello everybody, I am trying to apply the outlier and influence diagnostics as described by Viechtbauer and Cheung (2010) on a meta-analysis I conducted, using a four-level random-effects model with 528 included effect sizes. I have calculated the Cook's Distances and DFBETAS in R, using the functions for model diagnostics for rma.mv objects as described here: https://wviechtb.github.io/metafor/reference/influence.rma.mv.html However, I don't know how to decide about which studies are influential based on these values. I read the paper by Viechtbauer and Cheung, but I don't quite understand it. Is there some kind of formula to determine the threshold values of CD and DFBETAS for a meta-analytic model with k outcomes? Thanks in advance for your help. Best, Theresa [[alternative HTML version deleted]]
_______________________________________________ R-sig-meta-analysis mailing list @ R-sig-meta-analysis at r-project.org To manage your subscription to this mailing list, go to: https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
And to follow-up on this: There isn't any formula anyway. It's a bit like asking if there a formula that we can use to determine when a person is 'unusually tall' based on having measured people's height. It's all relative. We can just compare the height of the tallest person with that of the rest. If he/she sticks out (in the literal sense), then we might say that this person is quite tall compared to the rest. Therefore, even the rules mentioned here are essentially arbitrary: https://wviechtb.github.io/metafor/reference/influence.rma.uni.html Best, Wolfgang
-----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Michael Dewey Sent: Friday, 30 September, 2022 11:08 To: Th?lking, Theresa; r-sig-meta-analysis at r-project.org Subject: Re: [R-meta] Threshold values for Cook's Distances and DFBETAS Dear Theresa My feeling is that these diagnostics are better seen as identifying values which may benefit from further investigation. If I was concerned that they were unduly influential in the model I might try fitting leave-one-out models but since those models are essentially data-driven they are also only useful in an exploratory sense. So, even if there is a formula I would be sceptical about its value in answering the scientific question. Michael On 29/09/2022 21:35, Th?lking, Theresa wrote:
Hello everybody, I am trying to apply the outlier and influence diagnostics as described by
Viechtbauer and Cheung (2010) on a meta-analysis I conducted, using a four-level random-effects model with 528 included effect sizes. I have calculated the Cook's Distances and DFBETAS in R, using the functions for model diagnostics for rma.mv objects as described here: https://wviechtb.github.io/metafor/reference/influence.rma.mv.html
However, I don't know how to decide about which studies are influential based
on these values. I read the paper by Viechtbauer and Cheung, but I don't quite understand it. Is there some kind of formula to determine the threshold values of CD and DFBETAS for a meta-analytic model with k outcomes? Thanks in advance for your help.
Best, Theresa
Hello Wolfgang, hello Michael, thanks for taking the time to answer my question. This was very helpful. Best, Theresa
Von: Michael Dewey <lists at dewey.myzen.co.uk>
Gesendet: Freitag, 30. September 2022 11:07:31 An: Th?lking, Theresa; r-sig-meta-analysis at r-project.org Betreff: Re: [R-meta] Threshold values for Cook's Distances and DFBETAS Dear Theresa My feeling is that these diagnostics are better seen as identifying values which may benefit from further investigation. If I was concerned that they were unduly influential in the model I might try fitting leave-one-out models but since those models are essentially data-driven they are also only useful in an exploratory sense. So, even if there is a formula I would be sceptical about its value in answering the scientific question. Michael On 29/09/2022 21:35, Th?lking, Theresa wrote: > Hello everybody, > > > I am trying to apply the outlier and influence diagnostics as described by Viechtbauer and Cheung (2010) on a meta-analysis I conducted, using a four-level random-effects model with 528 included effect sizes. I have calculated the Cook's Distances and DFBETAS in R, using the functions for model diagnostics for rma.mv objects as described here: https://wviechtb.github.io/metafor/reference/influence.rma.mv.html > > > However, I don't know how to decide about which studies are influential based on these values. I read the paper by Viechtbauer and Cheung, but I don't quite understand it. Is there some kind of formula to determine the threshold values of CD and DFBETAS for a meta-analytic model with k outcomes? Thanks in advance for your help. > > > Best, > > Theresa > > [[alternative HTML version deleted]] > > _______________________________________________ > R-sig-meta-analysis mailing list @ R-sig-meta-analysis at r-project.org > To manage your subscription to this mailing list, go to: > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis > -- Michael http://www.dewey.myzen.co.uk/home.html