Message-ID: <61206fc1-c2d1-2f92-82f7-863b4d8a3576@uni-osnabrueck.de>
Date: 2022-01-07T15:38:58Z
From: Lukasz Stasielowicz
Subject: [R-meta] (Too) Many effect sizes for one single group
In-Reply-To: <mailman.3715.11.1641466801.5041.r-sig-meta-analysis@r-project.org>
Dear Catia,
under certain circumstances it could be a valid concern.
Fortunately, one can test it directly. One could conduct a sensitivity
analysis to examine the impact in your specific case: Do the results
(mean effect, standard error etc.) change much if you exclude certain
effect sizes?
Example:
Scenario 1: All effects are considered
Scenario 2: The study with "too many" effect sizes is excluded
Scenario 3: Only one or several effect sizes from the problematic study
are considered, e.g. by using the sample() function and choosing a
certain number of effects randomly. One could also repeat this procedure
to check the influence of the selection procedure.
If the estimates differ only slightly across the analyses then you could
proceed with the original idea (including all effects). You could
mention in the report that this decision is based on some sensitivity
analyses that you've conducted.
Best wishes
Lukasz
--
Lukasz Stasielowicz
Osnabr?ck University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstra?e 20
49074 Osnabr?ck (Germany)
Am 06.01.2022 um 12:00 schrieb r-sig-meta-analysis-request at r-project.org:
> Send R-sig-meta-analysis mailing list submissions to
> r-sig-meta-analysis at r-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> r-sig-meta-analysis-request at r-project.org
>
> You can reach the person managing the list at
> r-sig-meta-analysis-owner at r-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
>
>
> Today's Topics:
>
> 1. (Too) Many effect sizes for one single group
> (=?UTF-8?Q?C=C3=A1tia_Ferreira_De_Oliveira?=)
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 5 Jan 2022 19:32:29 +0000
> From: =?UTF-8?Q?C=C3=A1tia_Ferreira_De_Oliveira?= <cmfo500 at york.ac.uk>
> To: R meta <r-sig-meta-analysis at r-project.org>, "Viechtbauer,
> Wolfgang (SP)" <wolfgang.viechtbauer at maastrichtuniversity.nl>
> Subject: [R-meta] (Too) Many effect sizes for one single group
> Message-ID:
> <CACw+TffviaNdFERh3Pp361ZJVO2F11WG7S2G8WdWcONiHcusFg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Dear Wolfgang,
>
> I hope you had a lovely start to the year.
> I am sorry for starting the year with questions, but I just wanted to check
> whether there is any drawback from including a lot of effect sizes from a
> single paper when most labs contributed to the meta-analysis with just one
> or two effect sizes? This resulted in a dataset where half of the effect
> sizes come from multiple experiments run by the same group. The nested
> nature of the data and dependency of some effect sizes coming from the same
> participants is acknowledged in the model.
>
> Thank you!
>
> Catia
>