[R-meta] clubSandwich for variance components

Hi Stefanou,

This is a good question, about which there's currently limited evidence. robumeta can estimate two different working models, the correlated effects model and the hierarchical effects model:

robumeta's correlated effects model has a single variance component, tau-squared, estimated using the method of moments (MoM). Section 5.4.4 of the supplementary materials of my paper on "expanding the range of working models" (https://osf.io/nyv4u/) has some simulation results on the performance of the MoM estimator compared to the REML estimator of the more general CHE working model. We found that both estimators had similar accuracy (as measured by of root mean squared error) when the CE model is correctly specified or mildly mis-specified. (The MoM estimator is less biased but has higher variance than the REML estimator.) In the presence of within-study heterogeneity, however, the REML estimator from the more general CHE working model is more accurate--sometimes substantially so.

robumeta's hierarchical effects model has two variance components, tau-squared (between-study) and omega-squared (within-study), again estimated using MoM. We did not study this estimator in the paper, but my sense is that its performance would be similar to the MoM for the correlated effects model, or perhaps worse. These MoM estimators are not constructed in an efficient way and so will likely have higher variance than the REML estimators of the same working model. Admittedly, I'm speculating here, so it would definitely be helpful to have some simulation evidence with head-to-head comparisons between them (if anyone's lurking on the listserv looking for research ideas!).

James

On Mon, Aug 30, 2021 at 7:24 PM Stefanou Revesz <stefanourevesz at gmail.com> wrote:

Dear James,

Thank you very much. This is very helpful to know, almost any thread I
looked at in the archives talked about the systematic bias in the
estimates of variance components and the necessity of using RVE to
account for that, and yet I couldn't see how they are corrected. (this
can easily create a lot of confusion among substantively oriented
meta-analysts).

Just curious, do Tau and Omega estimates given by robumeta (I'm aware
of its underlying limitations in terms of modeling dependence etc.)
are themselves free from systematic bias (I think they are estimated
using closed-form equations)?

Thank you for this very important clarification,
Stefanou

On Mon, Aug 30, 2021 at 6:54 PM James Pustejovsky <jepusto at gmail.com> wrote:

Hi Stefanou,

clubSandwich provides standard errors, hypothesis tests, and confidence intervals for regression coefficient estimates (for meta-regressions and an array of other types of linear, generalized linear, and linear mixed effects models). It does *not* adjust variance component estimates (such as between-study heterogeneity estimates) not does it provide robust confidence intervals for variance components. The issues you asked about are challenging and relatively un-studied problems, and so unfortunately I don?t have any suggestions about alternative packages to look at.

James

On Aug 30, 2021, at 5:18 PM, Stefanou Revesz <stefanourevesz at gmail.com> wrote:

?Dear Colleagues,

I have recently learned about the RVE estimation and the usefulness of
the clubSandwich package in this regard.

I read the documentation of the package, but I was unable to determine
which function or functions in the package provide the adjusted
estimates of variance components and possibly their confidence
intervals (if I'm not mistaken, these are the ones that are often
unreliable if left unadjusted)?

Thank you for your assistance,
Stefanou