[R-meta] Covariance-variance matrix when studies share multiple treatment x control comparison
Dear Wolfgang, Thank you for your response and sorry I forgot to CC the mailing list! I am currently running my egger's regression test as shown below. My previous understanding was that I should look at the p-value of intercept term (following a previously published R code) if I run a "mixed" model using precision as moderator variable against residuals, but according to your comments I should be looking at the precision coefficients instead? So based on my outputs below, significance testing of plot asymmetry is at p=0.09 and not p=0.3823? Also, if I find significant violation of plot asymmetry in such case what additional options do I have to test these issues? I am currently calculating FSN which are extremely higher than proposed thresholds and removing influential outliers and re-fitting the model. But because rma.mv does not allow me to use other methods like trim and fill I wonder if these two other methods would be enough in case we detect plot asymmetry. Thank you for your time to answer these many questions. Best regards, JU
Full.egger.es<-rma.mv(hedged,var, method="REML", random = ~ 1 | Study, data=MHF) MHF$residuals<-residuals.rma(Full.egger.es) MHF$precision<-1/sqrt(MHF$var) egger.full<-rma.mv(residuals~precision,var,data=MHF,random = ~ 1 | Study) egger.full
Multivariate Meta-Analysis Model (k = 857; method: REML) Variance Components: estim sqrt nlvls fixed factor sigma^2 0.9929 0.9964 182 no Study Test for Residual Heterogeneity: QE(df = 855) = 4106.3487, p-val < .0001 Test of Moderators (coefficient(s) 2): QM(df = 1) = 2.7267, p-val = 0.0987 Model Results: estimate se zval pval ci.lb ci.ub intrcpt 0.0817 0.0936 0.8727 0.3828 -0.1017 0.2651 precision -0.0392 0.0238 -1.6513 0.0987 -0.0858 0.0073 . --- Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: Thursday, September 26, 2019 1:12 AM
To: Ju Lee <juhyung2 at stanford.edu>; r-sig-meta-analysis at r-project.org <r-sig-meta-analysis at r-project.org>; James Pustejovsky (jepusto at gmail.com) <jepusto at gmail.com>
Subject: RE: Covariance-variance matrix when studies share multiple treatment x control comparison
Sent: Thursday, September 26, 2019 1:12 AM
To: Ju Lee <juhyung2 at stanford.edu>; r-sig-meta-analysis at r-project.org <r-sig-meta-analysis at r-project.org>; James Pustejovsky (jepusto at gmail.com) <jepusto at gmail.com>
Subject: RE: Covariance-variance matrix when studies share multiple treatment x control comparison
Hi Ju, Glad to hear that you are making progress. Construction of the V matrix can be a rather tedious process and often requires quite a bit of manual work. I have little interested in generalizing fsn() for cases where V is not diagonal, because fsn() is more of interest for historical reasons, not something I would generally use in applied work. However, the 'Egger regression' test can be easily generalized to rma.mv() models. Simply include a measure of the precision (e.g., the standard error) of the estimates in your model as a predictor/moderator and then you have essentially a multilevel/multivariate version thereof (you would then look at the test of the coefficient for the measure of precision, not the intercept). I also recently heard a talk by Melissa Rodgers and James Pustejovsky (who is a frequent contributor to this mailing list) on some work in this area. Maybe he can chime in here. Best, Wolfgang -----Original Message----- From: Ju Lee [mailto:juhyung2 at stanford.edu] Sent: Thursday, 26 September, 2019 8:13 To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis at r-project.org Subject: Re: Covariance-variance matrix when studies share multiple treatment x control comparison Dear Wolfgang, I deeply appreciate your time looking into this issue, and this has been immensely helpful. I was able to incorporate all possible inter-dependence among effect sizes by adding different layers of non-independence to our dataframe. I manually calculated hedges'd based on based on Hedges and Olkin (1985), and it generates exactly same value as hedges' g in escalc() "SMD" function. So hopefully I am doing everything right using the equation we've discussed earlier. I have been also wondering if it is possible to account of this variance-covariance structure that I've constructed when running publication bias analysis, for example, when using fsn() function or modified egger's regression test (looking at intercept term of residual ~ precision meta-regression using rma.mv). I had no luck so far finding information on this, and I would appreciate if you have any suggestions related to this Thank you for all of your valuable helps! Best regards, JU