[R-meta] When to skip an extra level?
Dear Wolfgang, Tim Thank you for bringing such interesting topic. What do you think in using the profile likelihood (PL) exploration of the variance-covariance parameters to see if exist any difference between a) full model with 3 levels,? b) model with 2 levels (sample error and paper level) c) model with 2 levels (sample error and outcome level). The PL have been propose as a way to check the assumption that the likelihood is indeed identifiable, additionally it is implemented in metafor.
On 15/09/2021 20:06, Viechtbauer, Wolfgang (SP) wrote:
Dear Tim, The question generally is when it makes sense to leave out a level if the data could be regarded as having a hierarchical structure (which is modeled in terms of nested random effects along the lines of '~ 1 | var1/var2/var3/...') and if so, which level(s) to leave out. I don't think there is any general consensus on this or even much empirical evidence to back up any particular approach. However, in general, I would say that if the number of units at a particular level is very similar to the number of units at one level below it (e.g., there are 199 papers and 200 studies - so one paper describes two studies while the remaining 198 papers describe one study -- to make the example from the second link even more extreme), then it becomes very difficult to distinguish the variances at those two levels and I would consider dropping one of the two levels. I don't have any super strong feelings on whether to then drop the upper (paper) or lower (study) level -- in the extreme scenario above, it is unlikely to matter. Dropping the paper level would treat the two studies from that one paper as independent. Dropping the study level would assume that the average true effects (averaged over whatever lower levels there are in the hierarchy below 'studies') in those two studies from that one paper are homogeneous. Neither is (probably) correct. I cannot tell you where the exact point is (in terms of # of papers versus # of studies) where I would start to consider dropping a level. Best, Wolfgang
-----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Timothy MacKenzie Sent: Wednesday, 15 September, 2021 2:31 To: R meta Subject: [R-meta] When to skip an extra level? Dear Meta-analysis Community Members, I want to get some clarity regarding when not to add an additional level. I have found two posts and was wondering how they agree with one another? (It seems the first one says is at odds with the second one) ***This post:https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018- July/000896.html suggests that we should avoid adding an extra level (row id) in: random = ~ 1 | study/outcome/id if not so many "studies" have repeatedly used the same "outcome". ***This post:https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2019- March/001479.html (second message from the top) suggests that we should avoid adding an extra level (study_id) in: random = ~ 1 | paper_id/study_id/row_id Arguing that "One can probably skip a level if the number of units at a particular level is not much higher than the number of units at the next level (the two variance components are then hard to distinguish). So, for example, 200 "studies" in 180 "papers" is quite similar, so one could probably *leave out the studies* level and only add random effects for papers (the two variance components are then hard to distinguish)." Sincerely, Tim
_______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
Kind regards/Saludos cordiales *Philippe Tadger* ORCID <https://orcid.org/0000-0002-1453-4105>, Reseach Gate <https://www.researchgate.net/profile/Philippe-Tadger> Phone/WhatsApp: +32498774742 [[alternative HTML version deleted]]