[R-meta] Inflated confidence intervals

Dear Wasim,

you subscribed with address w.iqbal at ncl.ac.uk but you are posting from w.iqbal at newcastle.ac.uk. So, based on the address you are using for posting, you are not subscribed and hence the admins must approve each of your posts manually (which is a bit tedious). Please subscribe with the address that you are actually using for posting.

Back to the topic of your posts. Adding together all coefficients does not give you an 'adjusted intercept'. Let's consider a simpler example with two outcomes and use a dataset that comes with the metafor package, so we can actually run analyses and compare results. I'll use the Berkey et al. (1998) dataset for this; see help(dat.berkey1998).

### load data
dat <- get(data(dat.berkey1998))

### univariate model for outcome AL
rma(yi, vi, data=dat, subset=outcome=="AL")

### relevant part of the results
# estimate      se     zval    pval    ci.lb    ci.ub
#  -0.3457  0.0885  -3.9058  <.0001  -0.5191  -0.1722  ***

### univariate model for outcome PD
rma(yi, vi, data=dat, subset=outcome=="PD")

### relevant part of the results
# estimate      se    zval    pval   ci.lb   ci.ub
#   0.3606  0.0592  6.0905  <.0001  0.2445  0.4766  ***

# multivariate model
V <- lapply(split(dat[,c("v1i", "v2i")], dat$trial), as.matrix)
V <- bldiag(V)
res <- rma.mv(yi, V, mods = ~ outcome, random = ~ outcome | trial, struct="UN", data=dat)
res

### relevant part of the results
#            estimate      se     zval    pval    ci.lb    ci.ub
# intrcpt     -0.3392  0.0879  -3.8589  0.0001  -0.5115  -0.1669  ***
# outcomePD    0.6926  0.0744   9.3120  <.0001   0.5469   0.8384  ***

### compute 'intrcpt + outcomePD'
predict(res, 1)

### output
#    pred     se  ci.lb  ci.ub cr.lb cr.ub tau2.level
#  0.3534 0.0588 0.2381 0.4688    NA    NA         NA

So, in the multivariate model, the intercept corresponds to the estimate for outcome AL and the coefficient for outcomePD is the difference between outcome PD and outcome AL. What you are essentially doing is 'intrcpt + outcomePD' (i.e., adding all coefficients), which in this case gives the estimate for outcome PD. This is *not* the estimate for outcome AL that is somehow adjusted for outcome PD.

You are working with three outcomes, but the same applies. Adding all coefficients does not give you an 'adjusted intercept'.

Best,
Wolfgang

-----Original Message-----
From: Wasim Iqbal (UG) [mailto:W.Iqbal at newcastle.ac.uk]
Sent: Tuesday, 18 September, 2018 19:51
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: Inflated confidence intervals

ATTACHMENT(S) REMOVED: HNRC presentation------.pptx

Sorry I was informed by another member that they had to approve my emails each time (even though I have subscribed) so I had taken it out to prevent him from getting notifications.

My understanding is that the "intercept" is equivalent to the univariate model if you were to produce a model using rma.uni for only the first outcome (i.e. total cholesterol). Anything after it would be the "slopes"; The difference from the intercept.

In my example I had given you, the slopes would be HDL and LDL. Therefore, if we take into account HDL and LDL outcomes in a multivariate model. Then, compute the overall intercept for this model using predict(x,newmods=(1,1)) this would give us an "adjusted intercept". What I mean by adjusted is that we have taken into account the difference between HDL and LDL. Whereas, in a univariate model (rma.uni) this would be without taking into account the difference of HDL and LDL I would refer this to the "unadjusted model".

I think I may know where the confusion may be coming from. The outcomes I have included in my model are "individual measurements". I had converted these to "cohens D" so that we could include these in a single model (multivariate) and calculate the covariance between each etc.. In another example, I have combined body mass index (kg/m2), body weight (kg) and waist circumference (cm) by converting them all into "cohens D" so that they can be analysed in a single model (multivariate model). Then, I can change the order of the intercept to get the overall model for each outcome. Basically, these outcomes are not "groups in a set of data" but individual measurements.

Please refer to my presentation I had given to my research centre a few months ago which explains my methodology this show you what I mean.

Hope this makes a little more sense.

I referred to the multivariate model as "trivariate" because it had simply taken into account all "three outcomes".

Kind regards
Wasim

________________________________________
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org> on behalf of Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: 18 September 2018 18:14
To: Wasim Iqbal (UG)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Inflated confidence intervals

Please always cc the mailing list.

I don't know what you mean by 'trivariate mean' or 'adjusted mean' (adjusted for what?). If you want the average of the three outcomes, then it would be:

intrcpt + 1/3 * MeasurementHDL + 1/3 * MeasurementTC

so in this case:

-0.2376 + 1/3 * 0.1982 + 1/3 * -0.0148 = -0.1764667

which you can also get with: predict(resMV, newmods = c(1/3, 1/3)).

I don't know what you mean by 'Univariate model'. A model that does not distinguish between the three outcomes? Why would this model give an estimate that is equal to the intercept of the model whose output you showed?

Best,
Wolfgang

-----Original Message-----
From: Wasim Iqbal (UG) [mailto:W.Iqbal at newcastle.ac.uk]
Sent: Sunday, 16 September, 2018 17:18
To: Viechtbauer, Wolfgang (SP)
Subject: Re: Inflated confidence intervals

Sorry the difference for both HDL and LDL from the intercept would be....-0.0541. -0.0394 would be just for HDL

Wasim
________________________________________
From: Wasim Iqbal (UG)
Sent: 16 September 2018 16:13:16
To: Viechtbauer, Wolfgang (SP)
Subject: Re: Inflated confidence intervals

Sorry I am getting easily confused. Essentially, if you were to work out the trivariate mean (or adjusted mean) for this model would this be the difference of both HDL and LDL from the intercept (i.e. -0.0394)? Therefore, if I was to show this in a table with the univariate model this would be:

Univariate model: -0.2376
Trivariate model : -0.0394

I assumed that we must add the intercept to the slopes of HDL and LDL to get the overall adjusted mean? However, now having second thoughts based on what you said (and my textbooks) by adding the intercept ( i.e. -0.2376+0.1982+-0.0148) I am actually including the intercept again when it was already included? (silly me).

Thank you for clarifying. Its the simple things that catch me!

Kind regards
Wasim
________________________________________
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: 16 September 2018 14:30:01
To: Wasim Iqbal (UG)
Cc: r-sig-meta-analysis at r-project.org; Gavin Stewart; Chris Seal
Subject: RE: Inflated confidence intervals

I apparently haven't had enough coffee today, so first a correction on my part:

predict(resMV, newmods = c(1,0)) gives the estimated (average) outcome for HDL (i.e., -0.2376 + 0.1982 = -0.0394). The coefficient for HDL (i.e., 0.1982) is already the difference between HDL and LDL.

But you seem to be after something different. Apparently, you want to add the intecept and the two coefficients together, so: -0.2376 + 0.1982 + -0.0148 =~ -0.0541, which indeed you would obtain with predict(resMV, newmods = c(1,1)). But what is the meaning of this?

If you want a marginal mean, that is, the average of the three outcomes, then you would want:

intercept + 1/3 * HDL + 1/3 * TC

(assuming the intercept corresponds to LDL, as in the output you showed), which you would get with predict(resMV, newmods = c(1/3,1/3)). But maybe I am still misunderstanding.

Best,
Wolfgang

_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis