[R-meta] Meta-analysis of within-subject experimental designs with multiple treatment factors?

Dear Listmembers,

here is a repost of my earlier mail that I unfortunately posted in
html-format... I hope this time the data example is readable:

---

I am planning to conduct a meta-analysis based on data from a series of
factorial survey studies. The factorial survey studies I'd like to analyze
correspond to within-subject experimental designs with multiple treatment
factors. However, there seem to be very few, if any, meta-analyses using
data from factorial survey studies or related designs. I thus lack examples
corresponding to the design I need to ask some questions about below. So
here's a brief description of my data in general terms:

A factorial survey, also known as a vignette study or factorial vignette
design, is a research method used to study how individuals make judgments
or decisions in complex situations by presenting them with a series of
hypothetical scenarios (vignettes) that vary systematically across multiple
factors or attributes. For example, respondents might be asked how likely
they would vote for a candidate of a political party who is

- male vs. female, (factor 1)

- has a leftwing vs. rightwing orientation (factor 2)

- and behaves very empathic/somewhat empathic/not empathic (factor 3).

This results in a 2 x 2 x 3 factorial design with 12 different vignettes.

In the original data, the factor levels were measured using dummy
variables. The results are unstandardised regression slopes plus standard
errors. Accordingly, I'd like to meta-analyse 4 unstandardised regression
slopes:

- the slope of the dummy variable for factor A (the average effect of male
vs. female), fA_slope

- the slope of the dummy variable for factor B (the average effect of
rightwing vs. leftwing orientation), fB_slope

- the two slopes of the two dummy variables for factor C (somewhat
empathic vs very empathic and not empathic vs very empathic), fC_slope1 &
fC_slope2

Using one dependent variable, all studies were conducted as full or
fractional factorial within-subjects designs. Therefore, I believe it would
be best to account for any potential non-independence in the data, given
that each respondent rated multiple (up to 12) vignettes. What I find a bit
challenging is that there is not just one average effect to examine, but
four in total. Furthermore, these four average slopes result from three
orthogonal factors. I (think I) have been able to combine clubSandwich and
metafor to examine these data - but I would be very happy to hear if you
think my approach is adequate, or if there are alternative ways to analyse
data from such a research design?

Here's an illustration/sample of the data   restricted to 5 studies only,
the  real  dataset will comprise of approx. 20 studies. Notice that I
arranged all coefficient values in one column:

Data (df) for illustration:

study coeff_name     coeff_value  v       coeff_index

A        fA_slope1      -0.07        0.000113    1
A        fB_slope1      0.17         0.000344    2
A        fC_slope1      -0.08        0.000160    3
A        fC_slope2      -0.12        0.000171    4
B        fA_slope1      -0.08        0.000179    5
B        fB_slope1      0.18         0.000504    6
B        fC_slope1      -0.06        0.000309    7
B        fC_slope2      -0.14        0.000435    8
C        fA_slope1      -0.06        0.000097    9
C        fB_slope1      0.16         0.000319    10
C        fC_slope1      -0.07        0.000150    11
C        fC_slope2      -0.11        0.000152    12
D        fA_slope1      -0.03        0.000107    13
D        fB_slope1      0.15         0.000452    14
D        fC_slope1      -0.05        0.000198    15
D        fC_slope2      -0.10        0.000196    16
E        fA_slope1      -0.04        0.000123    17
E        fB_slope1      0.15         0.000366    18
E        fC_slope1      -0.11        0.000193    19
E        fC_slope2      -0.18        0.000205    20

Here's the code I used:

library(metafor)

library(clubSandwich)

# Create sampling variance covariance matrix

V_mat <- impute_covariance_matrix(Testdata$v, cluster = df$study, r = 0.6)

# model in metafor

mod <- rma.mv(coeff_value ~ 0 + coeff_name, V = V_mat,  #coeff_name
contains all coefficients as factor levels

              random = ~ 1 | study / coeff_index,   #coefficients are
nested in studies

              data = df)

# clustered SEs and CIs

conf_int(mod, vcov = "CR2")

And here are the results:

Coef.                                 Estimate      SE              d.f.
      Lower_95%CI Upper_95%CI
coeff_namecfA_slope1   -0.0553     0.00924     3.99     -0.081 -0.0296
coeff_namecfB_slope1   0.1623      0.0057       3.97      0.146 0.1782
coeff_namecfC_slope1   -0.0739     0.01044     3.97     -0.103 -0.0448
coeff_namecfC_slope2   -0.1289     0.01465     3.92      -0.17 -0.0879

The code seems to work. But as I outlined at the beginning, it is not
clear to if one might arrange the data in "long format" with all
coefficients of interest in one column as I did. I thus highly appreciate
your advice and any comments

Best,
Elmar


        [[alternative HTML version deleted]]