Skip to content

[R-meta] Specifying V in nested subgroup analysis (rma.mv + clubSandwich)

3 messages · Timothy MacKenzie, James Pustejovsky

Timothy MacKenzie
# 
Dear Meta SIG Members,

I'm running a nested subgroup analysis. That is:

1. Studies are subgrouped by "study_type" into standard vs. alternative.
2. Each previously made subgroup is further subgrouped by "reporting"
into subscale vs. composite (see data example below).

Effect sizes in each study are correlated (due to the individual study
designs) but my question is: given the "nested subgroup nature" of my
model, how should I specify the V (subgroup=NULL, or
subgroup=study_type, or subgroup=reporting)?

Thanks,
Tim M

(V <- with(dat1, impute_covariance_matrix(vi, study, r=.6,subgroup=NULL)))

g<-rma.mv(yi ~ 0 + study_type:reporting, V, random = list(~study_type
| study, ~interaction(study_type,reporting) | obs), struct =
c("DIAG","DIAG"), data = dat1)

m="
study subscale  reporting  obs include yi   vi         study_type
1        A      subscale   1   yes     1.94 0.33503768 standard
1        A      subscale   2   yes     1.06 0.01076604 standard
2        A      subscale   3   yes     2.41 0.23767389 standard
2        A      subscale   4   yes     2.34 0.37539841 standard
3        A&C    composite  5   yes     3.09 0.31349510 standard
3        A&C    composite  6   yes     3.99 0.01349510 standard
4        A&B    composite  7   yes     2.90 0.91349510 standard
4        A&B    composite  8   yes     3.01 0.99349510 standard
5        G&H    composite  9   yes     1.01 0.99910197 alternative
5        G&H    composite  10  yes     2.10 0.97910095 alternative
6        E&G    composite  11  yes     0.11 0.27912095 alternative
6        E&G    composite  12  yes     3.12 0.87910095 alternative
7        E      subscale   13  yes     0.08 0.21670360 alternative
7        G      subscale   14  yes     1.00 0.91597190 alternative
8        F      subscale   15  yes     1.08 0.81670360 alternative
8        E      subscale   16  yes     0.99 0.91297170 alternative"
dat1 <- read.table(text=m,h=T)
5 days later
James Pustejovsky
# 
Hi Tim,

If the predictor study_type is a study-level variable, then setting
subgroup = study_type will have no effect and should produce results
identical to using subgroup = NULL.

If you want to estimate average effect sizes for each reporting
category based only on the direct evidence (estimates from composite
scales contribute to the average for composite scales, estimates from
subscales contribute to the average for subscales), then set subgroup
= reporting.

On the other hand, if you set subgroup = NULL, then the average
effects for composite scales will be influenced a little bit by the
effect size estimates from subscales that co-occur in the same study
with estimates from composite scales, and the average effects for
subscales will be influenced a little bit by the effect size estimates
from composite scales that co-occur in the same study with estimates
from subscales.


James
On Thu, Dec 2, 2021 at 2:29 PM Timothy MacKenzie <fswfswt at gmail.com> wrote:
Timothy MacKenzie
# 
Dear James,

Thanks for your reply. I wanted my single model (gg) to be equivalent
to gg1 and gg2 (below). But if you use: print(gg, digits = 7);
print(gg1, digits = 7) ;  print(gg2, digits = 7), you realize the
variance components don't quite match.

Now, if you change the first random term in gg1 and gg2 from: ~1 |
study to: ~study_type | study, we get a better match.

Also, James, neither reporting levels nor study_type levels co-occur
in any study.

Thanks,
Tim M

gg1=rma.mv(yi ~ 0+reporting, V, random = list(~1 | study, ~ reporting | obs),
           struct = c("DIAG","DIAG"),
           subset = study_type=="alternative", data = dat1)

gg2=rma.mv(yi ~ 0+reporting, V, random = list(~1 | study, ~ reporting | obs),
            struct = c("DIAG","DIAG"),
            subset = include=="yes" & study_type=="standard", data = dat1)
On Tue, Dec 7, 2021 at 10:03 PM James Pustejovsky <jepusto at gmail.com> wrote: