[R-meta] The P value of correlation coefficent in meta-analysis
Hi Guido and Lukasz, Great thanks for the details! This issue was already extensively discussed in these publications, especially in the meta-analysis book. Thank you for helping working things out! Cheers! Pengzhen ---- Replied Message ---- | From | Lukasz Stasielowicz<lukasz.stasielowicz at uni-osnabrueck.de> | | Date | 5/8/2024 03:25 | | To | <r-sig-meta-analysis at r-project.org> | | Cc | <maiqi1317 at 163.com> | | Subject | Re: The P value of correlation coefficent in meta-analysis | Hi Pengzhen, Oh dear, your intuition is obviously correct. If we exclude non-significant correlations, then we will overestimate the correlation. In other words, we would get a biased effect estimate. This issue is addressed in many meta-analytic textbooks, e.g. Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2021). Introduction to meta-analysis. John Wiley & Sons. The authors offer a few free chapters, some of which could be useful in your case. https://introduction-to-meta-analysis.com/download/c01.pdf For example, Figure 1.1 (p. 4) clearly shows that studies with insignificant p-values are included in the meta-analysis. The book also contains a chapter, "Vote counting - a new name for an old problem," which has a nice example showing that even when all individual studies have large p-values, the meta-analytic estimate can be statistically significant. By combining the individual studies, we increase the power and can detect even small effects. Best, Lukasz -- Lukasz Stasielowicz Osnabr?ck University Institute for Psychology Research methods, psychological assessment, and evaluation Lise-Meitner-Stra?e 3 49076 Osnabr?ck (Germany) Twitter: https://twitter.com/l_stasielowicz Tel.: +49 541 969-7735
On 08.05.2024 03:30, r-sig-meta-analysis-request at r-project.org wrote:
Send R-sig-meta-analysis mailing list submissions to r-sig-meta-analysis at r-project.org To subscribe or unsubscribe via the World Wide Web, visit https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis or, via email, send a message with subject or body 'help' to r-sig-meta-analysis-request at r-project.org You can reach the person managing the list at r-sig-meta-analysis-owner at r-project.org When replying, please edit your Subject line so it is more specific than "Re: Contents of R-sig-meta-analysis digest..." Today's Topics: 1. Online course: Meta-analysis in R (info at physalia-courses.org) 2. The P value of correlation coefficent in meta-analysis (Pengzhen Huang) 3. Correcting gain effects in nested studies (Zhouhan Jin) 4. Re: Correcting gain effects in nested studies (James Pustejovsky) ---------------------------------------------------------------------- Message: 1 Date: Tue, 7 May 2024 21:06:47 +0200 (CEST) From: "info at physalia-courses.org" <info at physalia-courses.org> To: r-sig-meta-analysis at r-project.org Subject: [R-meta] Online course: Meta-analysis in R Message-ID: <1715108807.553710121 at webmail.jimdo.com> Content-Type: text/plain; charset="utf-8" Dear all, There are only 2 seats left for our upcoming online course, META-ANALYSIS IN R. Dates: May 13-16, 2024 Online: Accessible internationally This course covers: Systematic review and meta-analysis process Statistical analysis methods and interpretation Model diagnostics and sensitivity analyses Practical exercises with real meta-analytic datasets Prerequisites include basic statistical knowledge and familiarity with R. Resources for R preparation are provided. Course website: [ https://www.physalia-courses.org/courses-workshops/metain-r/ ]( https://www.physalia-courses.org/courses-workshops/metain-r/ ) Best regards, Carlo -------------------- Carlo Pecoraro, Ph.D Physalia-courses DIRECTOR info at physalia-courses.org mobile: +49 17645230846 ------------------------------ Message: 2 Date: Wed, 8 May 2024 03:47:15 +0800 (GMT+08:00) From: "Pengzhen Huang" <maiqi1317 at 163.com> To: =?UTF-8?Q?r-sig-meta-analysis=40r-pr=E2=80=A6?= <r-sig-meta-analysis at r-project.org> Subject: [R-meta] The P value of correlation coefficent in meta-analysis Message-ID: <505c373a.1c5.18f54988e70.Coremail.maiqi1317 at 163.com> Content-Type: text/plain; charset="utf-8" Dear Community, I submitted a meta-analysis paper months ago and am now dealing with the reviewers' comments. In my research, the Pearson correlation coefficients are considered as effect size and put into the meta-analysis, and we regard the coefficients representing to what extent two variables are correlated with each other. On this point, one of reviewers argues that "as not all r values are significant, it does not make sense to put these non-significant correlation coefficients into the analysis". I?m not sure how to reply to this reviewer?s comment. But I guess this may be a common issue in meta-analysis. May I have some advice from you or could you tell me some references I should read through? I would greatly appreciate any suggestions you can provide! All the best, Pengzhen ------------------------------ Message: 3 Date: Wed, 8 May 2024 00:14:11 +0000 From: Zhouhan Jin <zjin65 at uwo.ca> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org> Subject: [R-meta] Correcting gain effects in nested studies Message-ID: <d68c25db-a085-4e10-9e4b-d23de85a24ee at Spark> Content-Type: text/plain; charset="utf-8" Hello All, Hedges (2007) provides formulas for adjusting SMD effects (g) and their SEs for when primary studies have a nested design (below). But I want to compute gain effects (ex. SMCC in metafor::escalc) from my nested studies, not SMDs. So, how can I adjust my SMCCs and their SEs for nestedness in the primary studies? adjusted_g = g * sqrt(1 - ((2 * (n_bar - 1) * icc) / (n_cluster * n_bar - 2))) adjusted_SE = ((Nt+Nc)/(Nt*Nc))*(1 + ((n_bar- 1)*icc)) + ( g^2 * ( (((N_tot -2)*(1-icc)^2 ) + (n_bar*(N_tot - 2*n_bar)*icc^2) + (2* (N_tot - 2*n_bar) * icc * (1 - icc)) ) / ((2* (N_tot-2)) * ( (N_tot-2) - (2* (n_bar-1)*icc) )) ) ) Thanks a lot! Best wishes, Zhouhan ------------------------------ Message: 4 Date: Tue, 7 May 2024 20:30:33 -0500 From: James Pustejovsky <jepusto at gmail.com> To: Zhouhan Jin <zjin65 at uwo.ca> Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org> Subject: Re: [R-meta] Correcting gain effects in nested studies Message-ID: <CAFUVuJyQmjrJJMFz3EZHGT+rR76YqKr1Zv6AVbvFqLcf+v_AOg at mail.gmail.com> Content-Type: text/plain; charset="utf-8" See Taylor, Pigott, and Williams (2022; https://doi.org/10.3102/0013189X211051319) for how to handle cluster-randomized trials that involve gain scores or covariate adjustment. They provide a shiny app too. The technical details are also described in Appendix E of the What Works Clearinghouse handbook (Version 5; https://ies.ed.gov/ncee/WWC/Docs/referenceresources/Final_WWC-HandbookVer5_0-0-508.pdf). See pp. 173-174 The methods described in these sources are consistent with the "general recipe" for standardized mean difference estimates as described here: https://www.jepusto.com/alternative-formulas-for-the-smd/ James
On Tue, May 7, 2024 at 7:14?PM Zhouhan Jin <zjin65 at uwo.ca> wrote:
Hello All, Hedges (2007) provides formulas for adjusting SMD effects (g) and their SEs for when primary studies have a nested design (below). But I want to compute gain effects (ex. SMCC in metafor::escalc) from my nested studies, not SMDs. So, how can I adjust my SMCCs and their SEs for nestedness in the primary studies? adjusted_g = g * sqrt(1 - ((2 * (n_bar - 1) * icc) / (n_cluster * n_bar - 2))) adjusted_SE = ((Nt+Nc)/(Nt*Nc))*(1 + ((n_bar- 1)*icc)) + ( g^2 * ( (((N_tot -2)*(1-icc)^2 ) + (n_bar*(N_tot - 2*n_bar)*icc^2) + (2* (N_tot - 2*n_bar) * icc * (1 - icc)) ) / ((2* (N_tot-2)) * ( (N_tot-2) - (2* (n_bar-1)*icc) )) ) ) Thanks a lot! Best wishes, Zhouhan ------------------------------ Subject: Digest Footer _______________________________________________ R-sig-meta-analysis mailing list @ R-sig-meta-analysis at r-project.org To manage your subscription to this mailing list, go to: https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis ------------------------------ End of R-sig-meta-analysis Digest, Vol 84, Issue 10 ***************************************************