[R-meta] [External] RE: Question about escalc, proportion ES, and nested data

Fri, Feb 4, 2022 7:23 AM

You don't have to derive an effect size - the proportions you have *are* your 'effect size' (leaving aside whether some transformation thereof may be beneficial). The problem is that you can't really compute any sensible estimate of their sampling variances without either doing an analysis that would directly produce the corresponding SEs or deriving the distribution properties of these proportions based on statistical theory. One possibility would be to just use 1/n or something like that as a very rough approximation of the sampling variances and then apply (cluster) robust inference methods.

Best,
Wolfgang

________________________________________
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: Wednesday, February 2, 2022 4:17 AM
To: Harris, Jordan L <jordan-l-harris at uiowa.edu>; r-sig-meta-analysis at r-
project.org <r-sig-meta-analysis at r-project.org>
Subject: [External] RE: Question about escalc, proportion ES, and nested data

Dear Jordan,

Please see below for my responses.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Harris, Jordan L
Sent: Tuesday, 01 February, 2022 22:50
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Question about escalc, proportion ES, and nested data

Hi List Members,

I am a graduate student who is new to R and meta-analyses, and I have been
running into problems getting my code sorted out.

I am conducting a meta-analysis to explore how the structure of psychopathology
changes across childhood and adolescence. My effect size of interest is
represented by a proportion score that is conceptualized as ratio of variance
accounted for by a general factor, called "general_es" (i.e., general / general
+
specific). These data do not currently have a sampling variance, nor have
transformed effect sizes been calculated. I have 3 levels of nested data: Level
1
= "timepoint_id", Level 2 = "sample_id", Level 1 = "study_id" which account for
non-independence of data. Here, I will call my data file "dat."

? 1.? How should I structure the escalc command to derive a "yi" and "V" values
needed for the rma.mv analysis? Would my measure be "PLO"?

"PLO" is for binomial data, which is not what you appear to have. A logit
transformation may in itself be useful for proportions (however derived), but the
calculation of the sampling variance in escalc() assumes that each proportion was
calculated based on a random variable that follows a binomial distribution.

Ideally, one would need the standard errors of the proportions, which should come
from whatever method/model was used to obtain those proportions. Then one can use
the delta method to obtain the sampling variances of the logit-transformed
proportions.

Getting the covariance between sampling errors would be even more difficult
(multiple proportions obtained from the same sample will have non-zero
correlations between the sampling errors).

? 2.? Would this structure be acceptable: rma.mv(yi, vi, random = ~ 1 |
study_id/sample_id/timepoint_id, data=dat)?

Possibly, but it is impossible to answer this properly without further details.
For example, this model assumes constant correlation across timepoints,
regardless of how far they are apart.

And as noted above, this model would not account for non-independent sampling
errors.

Thanks,
Jordan

[R-meta] [External] RE: Question about escalc, proportion ES, and nested data

Thread (4 messages)