[R-meta] Which method to compute 95% confidence intervals around individual effect sizes (standardized mean differences) ?
Dear Wolfang,
thank you for your answer, methods 1 and 2 gave indeed the same results, sorry I did not ckeck correctly. Unfortunately confidence intervals computed from method 1 and 3 are different.
Sometimes they are very different one being positive and the other negative (which is my main problem).
Please find below an example from my data with problematic cases highlighted with *!!!*
(cilow1 and ciup1 are lower CI computed with method 1, whereas cilow3 and ciup3 are upper CI computed from method 3 (rma.mv model))
cilow1 cilow3 ciup1 ciup3
1: 0.316 0.600 3.721 3.436
2: 3.758 3.223 11.928 12.463
3: 1.103 1.744 5.284 4.643
4: 3.027 4.369 9.975 8.634
5: 3.029 3.882 9.980 9.127
6: 10.258 15.743 30.346 24.861
7: 11.055 14.887 32.647 28.815
8: 7.519 7.137 22.475 22.857
9: 36.817 31.138 107.667 113.346
10: -0.804 -1.088 2.032 2.317
11: 4.374 4.909 13.613 13.079
12: -0.581 -1.222 2.318 2.959
13: 1.175 -0.166 5.440 6.782 *!!!*
14: 1.921 1.068 7.166 8.019
15: 4.305 -1.180 13.422 18.907 *!!!*
16: 6.964 3.132 20.893 24.725
17: 7.929 8.311 23.649 23.267
18: 42.736 48.415 124.944 119.265
Looking specifically at rows 12 to 18, this is the same experiment testing increasing concentration of a chemical from row 12 to 18 with all compared to the same control. When I look to the raw data, confidence intervals provided with method 1 (cilow1) seem logical because there is an increasing effect of the chemical with increasing concentration, whereas the lack of effect indicated by cilow3 for line 15 seem very strange.
Based on this I think I will finally choose the method 1, but it would be very nice to understand why the model gives different confidence intervals.
Best wishes,
Dakis
----- Mail original -----
De: "Viechtbauer, Wolfgang (SP)" <wolfgang.viechtbauer at maastrichtuniversity.nl>
?: "Dakis-Yaoba OUEDRAOGO" <dakis-yaoba.ouedraogo at mnhn.fr>, "r-sig-meta-analysis" <r-sig-meta-analysis at r-project.org>
Envoy?: Dimanche 23 Mai 2021 19:06:01
Objet: RE: Which method to compute 95% confidence intervals around individual effect sizes (standardized mean differences) ?
Dear Dakis,
All three methods you listed should give you identical results.
Best,
Wolfgang
-----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Dakis-Yaoba OUEDRAOGO Sent: Tuesday, 18 May, 2021 17:00 To: r-sig-meta-analysis at r-project.org Subject: [R-meta] Which method to compute 95% confidence intervals around individual effect sizes (standardized mean differences) ? Dear all, I am gathering studies about effects of various chemicals on corals, and for every concentration of a chemical I computed the standardized mean differences for several outcomes. My final goal is to know for which maximal concentration of a chemical no significant effect is observed, and for which minimal concentration of a chemical a significant effect is observed. To get this I computed the 95% confidence intervals around the standardized mean differences to identify the effect sizes that are significantly/non significantly different from zero. I am quite confused about how to properly compute these condidence intervals. I could see 3 different types of CI : 1/ 95% CI assuming a normal distribution data$cilow <- data$yi - sqrt(data$vi)*qnorm(0.05/2, lower.tail = FALSE) data$ciup <- data$yi + sqrt(data$vi)*qnorm(0.05/2, lower.tail = FALSE) 2/ 95% Wald-type confidence intervals These are computed using the summary.escalc() function in metafor 3/ The confidence intervals computed from a multi-level model rma.mv where a variance-covariance matrix V is specified to take into account that I have several concentrations compared to the same control (Correction of Gleser & Olkin, I followed the tutorial in http://www.metafor- project.org/doku.php/analyses:gleser2009#quantitative_response_variable to compute V) mod <- rma.mv(yi=yi, V=V, mods= ~1, random= ~1 | ID_case, data=data, method="REML") With forest(mod) I can see the individual study 95%CI on the plot and I can get them with mod$yi - sqrt(mod$vi)*qnorm(0.05/2, lower.tail = FALSE) mod$yi + sqrt(mod$vi)*qnorm(0.05/2, lower.tail = FALSE) Because the method chosen to compute the 95% CI around the individual standardized mean differences will greatly influence the conclusions about the problematic chemical concentrations, I will greatly appreciate any help, comment or advise on my issue. Best wishes, Dakis