Skip to content

[R-meta] Cooks distance from cluster in multilevel models

2 messages · Martineau, Roger, Viechtbauer Wolfgang (STAT)

Martineau, Roger
# 
Dear metafor users,

I downloaded the data set (i.e., p154-dataset.csv) from  Assinka and Wibbelink (2016) available from http://www.tqmp.org/RegularArticles/vol12-3/p154/p154.pdf


3-level model with multiple moderators (Listing 15 in the publication):
+                             random =list(~ 1 | effectsizeID, ~ 1 | studyID),
+                             tdist=TRUE, data=dataset)
Multivariate Meta-Analysis Model (k = 100; method: REML)

  logLik  Deviance       AIC       BIC      AICc
 -63.375   126.750   138.750   154.136   139.694

Variance Components:

           estim   sqrt  nlvls  fixed        factor
sigma^2.1  0.085  0.292    100     no  effectsizeID
sigma^2.2  0.149  0.386     17     no       studyID

Test for Residual Heterogeneity:
QE(df = 96) = 609.357, p-val < .001

Test of Moderators (coefficient(s) 2:4):
F(df1 = 3, df2 = 96) = 6.414, p-val < .001

Model Results:

            estimate     se    tval   pval   ci.lb   ci.ub
intrcpt        0.466  0.107   4.346  <.001   0.253   0.678
pyear         -0.038  0.018  -2.077  0.040  -0.074  -0.002
typeovert     -0.204  0.139  -1.473  0.144  -0.479   0.071
typecovert    -0.709  0.191  -3.707  <.001  -1.089  -0.330
To plot Cook's Distance values for cluster studyID (17 levels) from model multiple moderator, I did:
+                            cluster=dataset$studyID,
+                            progbar=TRUE)
  |=======================================================| 100%
15
15
[cid:image001.jpg at 01D317FF.F01E6720]
1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 17
 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 17
Effect sizes are unique among studies. Study 16 is missing despite there are 16 effect sizes in study 16. Any clue for that?

Thanks in advance,

Roger :)

Roger Martineau, mv Ph.D.
Centre de recherche et de d?veloppement
sur le bovin laitier et le porc
Agriculture et agroalimentaire Canada/Agriculture and Agri-Food Canada
T?l?phone/Telephone: 819-780-7319
T?l?copieur/Facsimile: 819-564-5507
2000, Rue Coll?ge / 2000, College Street
Sherbrooke (Qu?bec)  J1M 0C8
Canada
roger.martineau at agr.gc.ca<mailto:roger.martineau at agr.gc.ca>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20170818/37782390/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 15907 bytes
Desc: image001.jpg
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20170818/37782390/attachment-0001.jpg>
Viechtbauer Wolfgang (STAT)
# 
Cook's distance is a 'leave-out' measure. So, an estimate/cluster is removed, the model is refitted, and the change in the fixed effects (compared to the 'full' model based on all data and scaled by the var-cov matrix of the fixed effects from the full model) is quantified. It may not be possible to fit the model if a particular estimate/cluster is removed (e.g., the model doesn't converge anymore, the model matrix is then no longer of full rank). You should then see an NA value for that estimate/cluster.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Martineau, Roger
Sent: Friday, August 18, 2017 16:08
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Cooks distance from cluster in multilevel models

Dear metafor users,

I downloaded the data set (i.e., p154-dataset.csv) from? Assinka and Wibbelink (2016) available from http://www.tqmp.org/RegularArticles/vol12-3/p154/p154.pdf

3-level model with multiple moderators (Listing 15 in the publication):
+ ????????????????????????????random =list(~ 1 | effectsizeID, ~ 1 | studyID), 
+ ????????????????????????????tdist=TRUE, data=dataset)
Multivariate Meta-Analysis Model (k = 100; method: REML)

? logLik? Deviance?????? AIC ??????BIC????? AICc? 
?-63.375?? 126.750?? 138.750?? 154.136?? 139.694? 

Variance Components: 

?????????? estim?? sqrt? nlvls? fixed??????? factor
sigma^2.1? 0.085? 0.292??? 100???? no? effectsizeID
sigma^2.2? 0.149? 0.386???? 17???? no?????? studyID

Test for Residual Heterogeneity: 
QE(df = 96) = 609.357, p-val < .001

Test of Moderators (coefficient(s) 2:4): 
F(df1 = 3, df2 = 96) = 6.414, p-val < .001

Model Results:

??????????? estimate???? se??? tval?? pval?? ci.lb?? ci.ub
intrcpt??????? 0.466? 0.107?? 4.346? <.001?? 0.253?? 0.678
pyear???????? -0.038? 0.018? -2.077? 0.040? -0.074? -0.002
typeovert???? -0.204? 0.139? -1.473? 0.144? -0.479?? 0.071
typecovert??? -0.709? 0.191? -3.707? <.001? -1.089? -0.330

To plot Cook?s Distance values for cluster studyID (17 levels) from model multiple moderator, I did:
+ ???????????????????????????cluster=dataset$studyID, 
+ ???????????????????????????progbar=TRUE)
? |=======================================================| 100%
15 
15
1? 2? 3? 4? 5? 6? 7? 8? 9 10 11 12 13 14 15 17 
?1? 2? 3? 4? 5? 6? 7? 8? 9 10 11 12 13 14 15 17 

Effect sizes are unique among studies. Study 16 is missing despite there are 16 effect sizes in study 16. Any clue for that?

Thanks in advance,

Roger ?