[R-meta] An issue with selmodel( type="step")
Cool, thanks. I presume enough of those updates were already in the metafor I downloaded last week (4.5-12), otherwise it wouldn't have run. Is there another update I should try? FYI, I tried selmodel(..., type="step", steps=(0.025)) with a similar simulation of 2500 meta-analyses, all estimates significant, but this time with small-moderate true heterogeneity (SD=1.5) rather than trivial-small (SD=0.5). It's still working, although not that well: some remaining bias; coverage is not very good, in spite of the wide confidence intervals, much wider than those before adjustment; overall it's better than PEESE. I then tried type="beta" with the first 100 of the simulations. Again it took more than 2 hours, and only half of the sims had confidence limits for the fixed effects. Overall beta would be better, if there was some way to coax it to give more confidence limits. Others have already noted how heterogeneity reduces the effectiveness of adjustment for publication bias, so it's obviously important to include study and subject characteristics as predictors to try to reduce heterogeneity to a trivial value. Will -----Original Message----- From: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org> On Behalf Of Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis Sent: Thursday, March 28, 2024 1:10 AM To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org> Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl> Subject: Re: [R-meta] An issue with selmodel( type="step") I am working on some updates to selmodel() and one of those changes is that the function now continues to run even if an interval contains no p-values. As discussed previously, the corresponding delta estimate will then either try to drift to 0 or to infinity. Results should be treated with caution (as noted in the output from the function). Best, Wolfgang
-----Original Message-----
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org>
On Behalf Of Will Hopkins via R-sig-meta-analysis
Sent: Tuesday, March 26, 2024 20:47
To: 'R Special Interest Group for Meta-Analysis'
<r-sig-meta-analysis at r- project.org>
Cc: Will Hopkins <willthekiwi at gmail.com>
Subject: [R-meta] An issue with selmodel( type="step")
Wolfgang, according to the documentation "there must be at least one
observed p-value within each interval to fit this model. If this is
not the case, an error will be issued." When I tried it with
steps=(0.025) for a simulated meta-analysis in which all estimates
were significant (p<0.05), it issued an error ("One or more intervals
do not contain any observed p-values"), but the analysis nevertheless
produced a result. When I ran it with 2500 such simulations, it
produced point estimates for the fixed effects with 2456 simulations,
and confidence limits with 2399. See below for a typical result. The
selection model results show 0.0000 for the estimated probability of
non-significant p values, as expected, so how is it able to make
adjustments?
Type="steps" seems to be about as good as type="beta" for bias and coverage with this particular set of study characteristics, but the beta type runs so slowly that I have only used it for 100 simulations for
comparison so far.
It took more than an hour, and only 51/100 produced confidence limits
for the fixed effects, so in these respects the steps type is
definitely better).
Will
Mixed-Effects Model (k = 22; tau^2 estimator: ML)
tau^2 (estimated amount of residual heterogeneity): 0.2541 (SE = 0.2288)
tau (square root of estimated tau^2 value): 0.5041
Test for Residual Heterogeneity:
LRT(df = 1) = 4.9738, p-val = 0.0257
Test of Moderators (coefficients 1:2):
QM(df = 2) = 94.2813, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
xxx$SexFemale 2.9331 0.3022 9.7046 <.0001 2.4359 3.4302 ***
xxx$SexMale 1.1294 0.7376 1.5312 0.1257 -0.0838 2.3425
Test for Selection Model Parameters:
LRT(df = 1) = 22.8488, p-val < .0001
Selection Model Results:
k estimate se zval pval ci.lb ci.ub
0 < p <= 0.025 22 1.0000 --- --- --- --- ---
0.025 < p <= 1 0 0.0000 NA NA NA NA NA
_______________________________________________ R-sig-meta-analysis mailing list @ R-sig-meta-analysis at r-project.org To manage your subscription to this mailing list, go to: https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis