Hello all, Many meta-analyses will take their smaller-than-they-would-have-hoped summary effect size and make it look bigger by using Rosenthal and Rubin's (1982) binomial effect size display: http://www.cognadev.com/publications/BESD_Rosenthal_and_Rubin_1982.pdf. I have always thought this is a misleading metric. Someone asked a question 5 years ago on CrossValidated about it, and I tried to answer it with a little simulation: https://stats.stackexchange.com/questions/24067/is-the-binomial-effect-size-display-besd-a-misleading-representation-of-effect (this post provides a good, short summary of the original paper and the metric, if you are unfamiliar with it). I'm more of a simulate-and-see-if-it-works type of person?is there anyone who understands more of the math behind *why* this metric might be misleading? Best, Mark
[R-meta] Binomial Effect Size Display?
3 messages · Mark White, Viechtbauer Wolfgang (STAT), Michael Dewey
3 days later
I don't have the time to really dig into this and since I hardly see any use of the BESD in practice, it seems like a non-issue to me. But this article seems highly pertinent: Hsu, L. M. (2004). Biases of success rate differences shown in binomial effect size displays. Psychological Methods, 9(2), 183-197. Some criticisms are also discussed in: Randolph, J. J., & Edmondson, R. S. (2005). Using the Binomial Effect Size Display (BESD) to present the magnitude of effect sizes to the evaluation audience. Practical Assessment, Research & Evaluation, 10(14). Best, Wolfgang
Wolfgang Viechtbauer, Ph.D., Statistician | Department of Psychiatry and Neuropsychology | Maastricht University | P.O. Box 616 (VIJV1) | 6200 MD Maastricht, The Netherlands | +31 (43) 388-4170 | http://www.wvbauer.com -----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Mark White Sent: Saturday, July 15, 2017 15:36 To: r-sig-meta-analysis at r-project.org Subject: [R-meta] Binomial Effect Size Display? Hello all, Many meta-analyses will take their smaller-than-they-would-have-hoped summary effect size and make it look bigger by using Rosenthal and Rubin's (1982) binomial effect size display: http://www.cognadev.com/publications/BESD_Rosenthal_and_Rubin_1982.pdf. I have always thought this is a misleading metric. Someone asked a question 5 years ago on CrossValidated about it, and I tried to answer it with a little simulation: https://stats.stackexchange.com/questions/24067/is-the-binomial-effect-size-display-besd-a-misleading-representation-of-effect (this post provides a good, short summary of the original paper and the metric, if you are unfamiliar with it). I'm more of a simulate-and-see-if-it-works type of person?is there anyone who understands more of the math behind *why* this metric might be misleading? Best, Mark
1 day later
Dear Mark There are some examples in the Cooper and Hedges handbook in a chapter by Rosentahl where clinical trials which were stopped early on ethical grounds because of the large size of effect are shown to have very small values of r. The BESD was, I imagine, a response to the use of r which gave a misleading result here. But, as Wolfgang said in another post, the BESD is hardly ever seen in the wild because nobody running a clinical study would usually use anything other than odds/risk ratios or mean differences. Michael
On 15/07/2017 14:35, Mark White wrote:
Hello all, Many meta-analyses will take their smaller-than-they-would-have-hoped summary effect size and make it look bigger by using Rosenthal and Rubin's (1982) binomial effect size display: http://www.cognadev.com/publications/BESD_Rosenthal_and_Rubin_1982.pdf. I have always thought this is a misleading metric. Someone asked a question 5 years ago on CrossValidated about it, and I tried to answer it with a little simulation: https://stats.stackexchange.com/questions/24067/is-the-binomial-effect-size-display-besd-a-misleading-representation-of-effect (this post provides a good, short summary of the original paper and the metric, if you are unfamiliar with it). I'm more of a simulate-and-see-if-it-works type of person?is there anyone who understands more of the math behind *why* this metric might be misleading? Best, Mark [[alternative HTML version deleted]]
_______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis --- This email has been checked for viruses by AVG. http://www.avg.com