Message-ID: <f4241eaa7bf647cca6efc0793a3e5117@UM-MAIL3214.unimaas.nl>
Date: 2019-09-24T10:59:38Z
From: Wolfgang Viechtbauer
Subject: [R-meta] Covariance-variance matrix when studies share multiple treatment x control comparison
In-Reply-To: <SN6PR02MB4336D7A4218F4A156FF2113CF78E0@SN6PR02MB4336.namprd02.prod.outlook.com>
Hi Ju,
The equation for the covariances applies to the standardized mean difference for two independent groups (so, what one could call 'classical' Cohen's d or Hedges' g if one uses the bias correction). It does not apply to other types of standardized mean differences, such as those for dependent/paired groups. Also, the equation assumes that the pooled variance 'sdp' (that is used for computing pairwise SMDs, such as (m1 - m3) / sdp and (m2 - m3) / sdp for three groups, with group 3 being the reference group) has been computed by pooling the SDs of all three groups. Doing so isn't standard practice and takes some extra effort to implement.
Just a sidenote: When pasting tables like this, please use a fixed width font (and don't use tabs). Without that, the formating is all off and makes the table hard to read.
Even study 1 is much more complicated. It seems that rows 1 and 2 share a common group (mc=0.68 with nc=6). Then rows 3 and 4 share a common group (mt=0.84 with nt=9). But rows 1 and 4 and 2 and 3 also share a common group each. So, the var-cov matrix for this study would look like this:
[ 0.370 cov12 0 cov14 ]
[ 0.278 cov23 0 ]
[ 0.234 cov34 ]
[ 0.291 ]
In study 2, it would be:
[ 0.056 cov12 0 ]
[ 0.070 cov23 ]
[ 0.052 ]
Leaving aside the issue of whether SDs are getting pooled across all groups within a study or not, the covariance terms should be computable with the equation (19.19) from Gleser and Olkin (2009). I am not 100% sure though if the fact that the reference group is sometimes T and sometimes C matters. In any case, you should treat the group (whether it is actually a 'treatment' or a 'control' group as the 'control' group in equation (19.19)).
As for your ID variable and how to code it -- a single ID variable cannot capture these complexities. You will need 2 ID variables, indicating which group is the T and which is the C group. So, for study 1, it would be:
Study mt mc nt nc TID CID hedges.d var
1 0.92 0.68 6 6 1 3 0.937 0.370
1 0.69 0.68 9 6 2 3 0.038 0.278
1 0.84 0.69 9 9 4 2 0.659 0.234
1 0.84 0.92 9 6 4 1 -0.623 0.291
Best,
Wolfgang
-----Original Message-----
From: Ju Lee [mailto:juhyung2 at stanford.edu]
Sent: Thursday, 19 September, 2019 1:24
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis at r-project.org
Subject: Re: Covariance-variance matrix when studies share multiple treatment x control comparison
Dear Wolfgang and all,
Please ignore my previous e-mail,? and refer to this one instead:
Thank you very much for your response. I am using Hedges' d and I assumed this code applies to all SMD type of ES including Hedges' d?
I am still little baffled how I can apply this to more complex combinations. Below table looks busy, but I would appreciate if you can answer the following two questions. FYI, C_ID1 is the shared group ID.
1) In study 1, treatment 0.84 is shared in 3th and 4th response. To account for this inter-dependence, should I use "nt"? instead of "nc" for calculating cov.d matrix?
2) I am not sure how to proceed if treatments and controls are shared in fashion as study 2. Clearly, 1st/2nd response shares treatment (1.00) and 2nd/3rd shares? control (2.00), but 1st and 3rd do not share any experimental unit. I've categorized 2nd and 3rd with same C_ID1 because they share control, but am unclear if I should assign 1st response the same ID as 2nd and 3rd (c3) since it is still technically not independent from 2nd response (shared control) OR do I?assign it a complete different control ID?
Study?? mt? ? ?mc? ? ?nt? ? nc? ?C_ID1?? hedges.d? ? var
1?????? 0.92??? 0.68??? 6?????? 6?????? c1????? 0.937? ? ? ?0.370
1?????? 0.69??? 0.68??? 9?????? 6?????? c1????? 0.038? ? ? ?0.278
1?????? 0.84??? 0.69??? 9?????? 9?????? c2????? 0.659? ? ? ?0.234
1?????? 0.84??? 0.92??? 9?????? 6?????? c2????? -0.623? ? ? 0.291
2?????? 1.00? ?0.32? ? 10? ? ?20? ? ? c?? ? ?0.760? ? ? ?0.056
2?????? 1.00? ?2.00? ? 10? ? ? 5? ? ? ?c3 ? ? 0.885? ? ? ?0.070
2?????? 0.61? ?2.00? ? 15? ? ? 5? ? ? ?c3? ? ? 0.209? ? ? ?0.052
I hope I am not asking too much of your time, but if you can answer this specific question, it will clarify all the doubts I have with this stage of analysis.
Thank you very much, and I hope to hear from you.
Best wishes,
JU