[R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11
Thank you, James. I understand that the moments estimators might be more useful when working with small sample sizes. In my data set I have about 60 studies and 110 effect sizes. So as such the dataset is not small. But I do want to estimate effect sizes for smaller sets of the data (there are multiple set of interventions which can be distinguished). In the smaller sets the number of effects decreases to as low as 15-30. In this context, I thought DL might be a better estimator. I will look into the robumeta package. I also have a theoretical question around RVE. The estimates that I get for RVE have much higher standard errors compared to the DL/REML estimator. I understand that this is to be expected, RVE is also likely to result in higher Type I errors. Is there any way to control for that in the metafor package? Best Tarun Tarun Khanna PhD Researcher Hertie School Friedrichstra?e 180 10117 Berlin ? Germany khanna at hertie-school.org ? www.hertie-school.org<http://www.hertie-school.org/>
From: James Pustejovsky <jepusto at gmail.com>
Sent: 21 April 2020 16:22:38
To: Tarun Khanna
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11
Sent: 21 April 2020 16:22:38
To: Tarun Khanna
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11
Hi Tarun, Good question! The Dersimonian-Laird variance estimator is in a general class of what are called moment estimators. In principle, it is possible to use moment estimation for models that are more complex than the basic meta-analysis/meta-regression model (as estimated with rma.uni, for instance). In fact, this is precisely what the robumeta package does. The correlated effects model and hierarchical effects model implemented in that package use moment estimators of the between-study variance component, and for the hierarchical model, also the within-study variance component. As far as I understand them, I think these estimators are not as precise as using ML/REML, and are mainly intended as way to get "quick-and-dirty" values for use in a working model (which need not be super accurate, if RVE is then used to get standard errors/confidence intervals). There has also been some statistical work on moment estimators for more complex multi-variate models: * Chen, H., Manning, A. K., & Dupuis, J. (2012). A method of moments estimator for random effect multivariate meta?analysis. Biometrics, 68(4), 1278-1284. I'm not sure if the methods described here are implemented in software though. Kind Regards, James On Tue, Apr 21, 2020 at 7:46 AM Tarun Khanna <khanna at hertie-school.org<mailto:khanna at hertie-school.org>> wrote: Thanks for the excellent interpretation of RVE. I was also wondering if it's possible to use DL method with RVE estimation in R? Obviously one can use this with the rma function but I cannot see similar options for the rma.mv<http://rma.mv> function, which only allows "REML" or "ML" as the methods option. Is there any theoretical reason why we cannot calculate DL with RVE? Or is it just that the functionality is not built into rma.mv<http://rma.mv> function in metafor? Best Tarun Tarun Khanna PhD Researcher Hertie School Friedrichstra?e 180 10117 Berlin ? Germany khanna at hertie-school.org<mailto:khanna at hertie-school.org> ? www.hertie-school.org<http://www.hertie-school.org><http://www.hertie-school.org/> ________________________________ From: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org<mailto:r-sig-meta-analysis-bounces at r-project.org>> on behalf of r-sig-meta-analysis-request at r-project.org<mailto:r-sig-meta-analysis-request at r-project.org> <r-sig-meta-analysis-request at r-project.org<mailto:r-sig-meta-analysis-request at r-project.org>> Sent: 18 April 2020 12:00:02 To: r-sig-meta-analysis at r-project.org<mailto:r-sig-meta-analysis at r-project.org> Subject: R-sig-meta-analysis Digest, Vol 35, Issue 11 Send R-sig-meta-analysis mailing list submissions to r-sig-meta-analysis at r-project.org<mailto:r-sig-meta-analysis at r-project.org> To subscribe or unsubscribe via the World Wide Web, visit https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis or, via email, send a message with subject or body 'help' to r-sig-meta-analysis-request at r-project.org<mailto:r-sig-meta-analysis-request at r-project.org> You can reach the person managing the list at r-sig-meta-analysis-owner at r-project.org<mailto:r-sig-meta-analysis-owner at r-project.org> When replying, please edit your Subject line so it is more specific than "Re: Contents of R-sig-meta-analysis digest..." Today's Topics: 1. Re: Robust variance estimation (James Pustejovsky) ---------------------------------------------------------------------- Message: 1 Date: Fri, 17 Apr 2020 12:47:11 -0500 From: James Pustejovsky <jepusto at gmail.com<mailto:jepusto at gmail.com>> To: Emily Russell <emilyrussell99 at outlook.com<mailto:emilyrussell99 at outlook.com>> Cc: "r-sig-meta-analysis at r-project.org<mailto:r-sig-meta-analysis at r-project.org>" <r-sig-meta-analysis at r-project.org<mailto:r-sig-meta-analysis at r-project.org>> Subject: Re: [R-meta] Robust variance estimation Message-ID: <CAFUVuJwOX6uORO7BUxdyEtq+Mw8P3UiSRe9mDmYYdJ0u8+e=rg at mail.gmail.com<mailto:rg at mail.gmail.com>> Content-Type: text/plain; charset="utf-8" Hi Emily, I think one useful intuition about robust variance estimation is that its a way of capturing the uncertainty in an estimate *using only between-study variation*. The RVE approach is analogous to how one would calculate the standard error of a mean from a simple random sample, so it's helpful to review that first. Say that we have a random sample of N observations, Y1, Y2,...., YN, and we're trying to estimate the population mean from this sample. The usual estimator is the sample mean, Y-bar. The standard error of Y-bar is SE = SD / sqrt(N), where SD is the standard deviation of the sample observations. Okay so now let's think about the meta-analysis context. Say that you have a meta-analysis with multiple effect sizes, which could be correlated, nested within a set of studies, which can be treated as independent of each other. And say that our goal is just to estimate the population mean effect size across the set of studies. The alternative to RVE is to use a "model-based" approach to uncertainty estimation (like a multi-variate or hierarchical model). To do that properly, we have to come up with an appropriate model for how the effect sizes are related to each other (i.e., how they correlate) within each study, and then also how they vary across studies. In other words, we have to have a model for both the within-study variation (and covariation) and the between-study variation. We use this model for the within- and between-study variation to determine how to take a weighted average of all of the effect sizes. And then, in the model-based approach, we also use it to determine a standard error for the weighted average. As a result, the accuracy of the standard error *is contingent on the modeling assumptions being appropriate*. The RVE approach still uses a model to determine how to take a weighted average of all of the effect sizes, but it does not rely on the model for assessing the uncertainty of the average. Instead, it just uses the between study variation. Behind the RVE formulas are really two steps of calculation. First is to calculate an average effect size for each study. Since studies are independent, each of these average effects can be treated as independent. And the overall average is just an average of the study-specific average effect sizes (albeit with weights involved). So actually, we're in a situation that's very similar to taking the mean of a simple random sample, only now our "observations" are study-specific average effect size estimates. Consequently, the second step in the RVE standard error calculation is to take the SD of the study-specific average effect sizes, then dividing the square root of the number of studies. (Again, in practice there's weights involved, but the intuition is still the same.) There are two key advantage of this approach. One is that it works fine for most any set of weights we might use in calculating the overall average effect size. The weights don't have to be exactly right or optimal in any sense. The second is that we can do these calculations without knowing exactly how the individual effect sizes within each study are correlated with each other. All we need is to be able to calculate study-specific average effect sizes. So we don't have to rely on our modeling assumptions being exactly right/accurate in order to trust the standard errors from RVE. The intuition about using only between-study variation can actually be carried further to more complex scenarios with meta-regression on a set of covariates, too. Kind Regards, James On Fri, Apr 17, 2020 at 5:54 AM Emily Russell <emilyrussell99 at outlook.com<mailto:emilyrussell99 at outlook.com>> wrote: > Dear Friends and Colleagues > > I hope this is not too basic a question; but could someone give me an > intuitive rather than technical explanation of what robust variance > estimation does (as in robu in robumeta and robust in metafor)? I have > looked at the papers referred to but they are a bit 'heavy' for me. > > Thank you so much > > Emily > > [[alternative HTML version deleted]] > > _______________________________________________ > R-sig-meta-analysis mailing list > R-sig-meta-analysis at r-project.org<mailto:R-sig-meta-analysis at r-project.org> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis > [[alternative HTML version deleted]] ------------------------------ Subject: Digest Footer _______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org<mailto:R-sig-meta-analysis at r-project.org> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis ------------------------------ End of R-sig-meta-analysis Digest, Vol 35, Issue 11 *************************************************** [[alternative HTML version deleted]] _______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org<mailto:R-sig-meta-analysis at r-project.org> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis