[R-meta] (Too) Many effect sizes for one single group - R-SIG-meta-analysis

Mon, Jan 10, 2022 3:13 AM #

Dear Catia,

aggregating (computing average effect for a particular study) could 
conceal heterogeneity between effect sizes (e.g. r1 = -.20, r2= +.20 --> 
r_mean = 0). It could also impact moderator analyses (less effect sizes 
to test moderators that explain within-study variability).

Of course one could also directly test the impact of aggregating on 
heterogeneity estimates (e.g. tau), but in general I would suggest 
keeping as many effect sizes as possible by using other sensitivity 
analyses (see previous message).


Best wishes
Lukasz

Lukasz Stasielowicz
Osnabr?ck University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstra?e 20
49074 Osnabr?ck (Germany)

Am 08.01.2022 um 12:00 schrieb r-sig-meta-analysis-request at r-project.org:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis at r-project.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request at r-project.org
> 
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner at r-project.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
> 
> 
> Today's Topics:
> 
>     1. Re: (Too) Many effect sizes for one single group
>        (Lukasz Stasielowicz)
>     2. Re: (Too) Many effect sizes for one single group
>        (=?UTF-8?Q?C=C3=A1tia_Ferreira_De_Oliveira?=)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Fri, 7 Jan 2022 16:38:58 +0100
> From: Lukasz Stasielowicz <lukasz.stasielowicz at uni-osnabrueck.de>
> To: r-sig-meta-analysis at r-project.org
> Cc: cmfo500 at york.ac.uk
> Subject: Re: [R-meta] (Too) Many effect sizes for one single group
> Message-ID: <61206fc1-c2d1-2f92-82f7-863b4d8a3576 at uni-osnabrueck.de>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> Dear Catia,
> 
> under certain circumstances it could be a valid concern.
> Fortunately, one can test it directly. One could conduct a sensitivity
> analysis to examine the impact in your specific case: Do the results
> (mean effect, standard error etc.) change much if you exclude certain
> effect sizes?
> 
> Example:
> Scenario 1: All effects are considered
> Scenario 2: The study with "too many" effect sizes is excluded
> Scenario 3: Only one or several effect sizes from the problematic study
> are considered, e.g. by using the sample() function and choosing a
> certain number of effects randomly. One could also repeat this procedure
> to check the influence of the selection procedure.
> 
> If the estimates differ only slightly across the analyses then you could
> proceed with the original idea (including all effects). You could
> mention in the report that this decision is based on some sensitivity
> analyses that you've conducted.
> 
> 
> Best wishes
> Lukasz