[R-meta] effect size calculation with a null standard deviation

Fri, Feb 19, 2021 2:31 AM

Conceivable in the sense of "one can think it up"? Sure, you just did :)

Conceivable in the sense of "credible"? I don't think anybody knows.

It sounds to me though as if you are dealing with count data. Maybe measures such as standardized mean differences and response ratios (which are meant to be used for continuous variables) are not the most suitable choice in the first place. For counts, one could also consider using incidence rates (and differences/ratios thereof).

Best,
Wolfgang

________________________________________
De: "James Pustejovsky" <jepusto at gmail.com>
?: "Wolfgang Viechtbauer, SP" <wolfgang.viechtbauer at maastrichtuniversity.nl>
Cc: "Marianne DEBUE" <marianne.debue at mnhn.fr>, "r-sig-meta-analysis" <r-sig-meta-
analysis at r-project.org>
Envoy?: Jeudi 18 F?vrier 2021 20:15:40
Objet: Re: [R-meta] effect size calculation with a null standard deviation

Hi Marianne,
I agree with Wolfgang that this is an unusual case and that figuring out how to
handle it requires some contextual judgements, which folks on the mailing list
aren't really in a position to advise on. That said, here are two avenues that you
might like to investigate further:

With only 3 observations in a given group, using a sample standard deviation to
calculate a standardized mean difference for the group will give an EXTREMELY
noisy estimate. Pooling the pre-test variance across groups (as in the Morris
"dppc2" estimator) might mitigate the problem a bit--especially if only one of the
groups is small and the other is larger.

More broadly, the fact that you're encountering these sorts of situations makes me
wonder whether it would be better to move to a different effect size metric, such
as the response ratio (Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999). The
meta?analysis of response ratios in experimental ecology.?Ecology,?80(4), 1150-
1156.). This might be a good way to go if all of your outcomes are ratio scale
measurements. My understanding is that this is pretty common in ecology.

James


On Thu, Feb 18, 2021 at 2:49 AM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
Dear Marianne,

I suspect that you did not receive any responses, since what you are describing
are really unusual cases which have not been discussed in the literature (as far
as I know).

I think you will just have to make a decision yourself how to handle these cases
and be transparent about how you handled them.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Marianne DEBUE
Sent: Thursday, 21 January, 2021 15:57
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] effect size calculation with a null standard deviation

Hi everyone,

I'm conducting a meta-analysis in ecology. I'm using Morris "dpcc1" formulas ( [
https://journals.sagepub.com/doi/10.1177/1094428106291059 |
https://journals.sagepub.com/doi/10.1177/1094428106291059 ] ) to calculate the
effect size and its variance.

The effect size calculation implies a difference between post- and pre-Mean which
is then divided by the pre-Standard deviation.
I was wondering how to deal with studies which have pre-Standard deviation = 0
(leading to a division by 0) ?
How to deal with studies which have pre-Mean = post-Mean and a pre-Standard
deviation = 0 (leading to 0 divided by 0) ? If pre-Mean = post-Mean , can we
consider that the effect size is null, whatever the pre-Standard deviation ?

The variance calculation implies a division by (N - 3) (N : sample size).
How to deal with studies which have N = 3 ( leading to a division by 0) ?

Thank you for your help,

Marianne

[R-meta] effect size calculation with a null standard deviation

Thread (4 messages)