[R-meta] Negative r value for effect sizes within studies

I should add that it is entirely "possible" to encounter outcomes that
are measured in opposite directions across studies and that can be
meaningfully part of the same multivariate meta-analysis. But like I
mentioned, syncing the outcomes' directions often resolves the issue.

Here is a substantive example.

Studies that research the efficacy of teacher's feedback on students'
(English Language Learners) writing competence often focus on various
related linguistic features (e.g., verbs, articles etc.). However, due
to the debates over how these linguistic features ought to be
measured, these studies tend to measure some linguistic features "by
tradition" using the number of errors, but others using the number of
correct usage (I'm simplifying the actual measures to make this
discussion more general, but the directions of outcomes run opposite
to one another for different linguistic features).

The fact that these outcome levels are related is of great interest,
hence motivating a multivariate meta-analysis. On the other hand,
almost always these studies are multi-group to compare different types
of feedback and longitudinal to measure the durability of feedback's
effect on developing students' writing competence (to make matters
even more complex, they even use more than one comparison/control
group).

In a situation like what I described above (and more generally as
described in my previous email), if not synced, these opposing
outcomes could create a difficult situation where each study could
have a combination of positively and negatively correlated estimates
of effect size in it.

So, synching the outcome directions, at least for me, is inevitable.

Reza

On Fri, Sep 3, 2021 at 12:19 PM James Pustejovsky <jepusto at gmail.com> wrote:

To add to Wolfgang's comments, yes it is possible in principle. However, on
a practical level, it seems like it would be a very odd situation where you
would include two outcomes in the same meta-analysis that are *negatively*
correlated with each other. Say that one intervention study reports a
negative effect size estimate for depression and a positive effect size for
positive affect, two outcomes which are negatively correlated. It would not
make much practical sense to average the negative ES for depression and the
positive ES for positive affect together, which suggests that it would be
very odd to include both in a single meta-analysis.

On Fri, Sep 3, 2021 at 12:09 PM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

Dear Stefanou,

Would be unusual, but in principle, yes, it can. However, only certain
values of r are actually possible. Let's say there is a study with 5
estimates and you assume r = -0.6. Then the correlation matrix for this
study would be:

v <- matrix(-.6, 5, 5); diag(v) <- 1

which is not positive definite, which we can see by examining its
eigenvalues:

round(eigen(v)$values, 4)

(note the last eigenvalue is negative). The largest negative correlation
possible when p=5 (to get at least a positive semi-definite matrix) is
-0.25:

v <- matrix(-.25, 5, 5); diag(v) <- 1
round(eigen(v)$values, 4)

(the last eigenvalue is now essentially 0).

More generally, the lower bound is:

-1/(p-1)

So, you need to determine which study has the largest number of estimates
and based on that, you know what the lower bound for r is (at least, when
we assume that the correlation matrix is compound symmetric - as above).

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:

r-sig-meta-analysis-bounces at r-project.org] On

Behalf Of Stefanou Revesz
Sent: Friday, 03 September, 2021 18:47
To: R meta
Subject: [R-meta] Negative r value for effect sizes within studies

Dear All,

This may be too basic to ask. But can the r value chosen to represent
the common correlation among effect sizes within studies be a negative
one?

For example, is there a situation where I would need to use the following?

impute_covariance_matrix(..., r = -.6)

Thank you,
Stefanou

        [[alternative HTML version deleted]]