Skip to content

[R-meta] (Too) Many effect sizes for one single group

2 messages · Lukasz Stasielowicz, Cátia Ferreira De Oliveira

#
Dear Catia,

under certain circumstances it could be a valid concern.
Fortunately, one can test it directly. One could conduct a sensitivity 
analysis to examine the impact in your specific case: Do the results 
(mean effect, standard error etc.) change much if you exclude certain 
effect sizes?

Example:
Scenario 1: All effects are considered
Scenario 2: The study with "too many" effect sizes is excluded
Scenario 3: Only one or several effect sizes from the problematic study 
are considered, e.g. by using the sample() function and choosing a 
certain number of effects randomly. One could also repeat this procedure 
to check the influence of the selection procedure.

If the estimates differ only slightly across the analyses then you could 
proceed with the original idea (including all effects). You could 
mention in the report that this decision is based on some sensitivity 
analyses that you've conducted.


Best wishes
Lukasz
#
Dear Lukasz,

Thank you for your response! I have also opted for aggregating the results
to check if that study came out as an influential/outlier study and it did
not, would this also be a good approach?
I will also run the models in the way suggested. Thank you again.

Best wishes,

Catia

On Fri, 7 Jan 2022 at 15:38, Lukasz Stasielowicz <
lukasz.stasielowicz at uni-osnabrueck.de> wrote: