-----Original Message-----
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org> On Behalf
Of Will Hopkins via R-sig-meta-analysis
Sent: Tuesday, March 26, 2024 20:47
To: 'R Special Interest Group for Meta-Analysis' <r-sig-meta-analysis at r-
project.org>
Cc: Will Hopkins <willthekiwi at gmail.com>
Subject: [R-meta] An issue with selmodel( type="step")
Wolfgang, according to the documentation "there must be at least one
observed p-value within each interval to fit this model. If this is not the
case, an error will be issued." When I tried it with steps=(0.025) for a
simulated meta-analysis in which all estimates were significant (p<0.05), it
issued an error ("One or more intervals do not contain any observed
p-values"), but the analysis nevertheless produced a result. When I ran it
with 2500 such simulations, it produced point estimates for the fixed
effects with 2456 simulations, and confidence limits with 2399. See below
for a typical result. The selection model results show 0.0000 for the
estimated probability of non-significant p values, as expected, so how is it
able to make adjustments?
Type="steps" seems to be about as good as type="beta" for bias and coverage
with this particular set of study characteristics, but the beta type runs so
slowly that I have only used it for 100 simulations for comparison so far.
It took more than an hour, and only 51/100 produced confidence limits for
the fixed effects, so in these respects the steps type is definitely
better).
Will
Mixed-Effects Model (k = 22; tau^2 estimator: ML)
tau^2 (estimated amount of residual heterogeneity): 0.2541 (SE = 0.2288)
tau (square root of estimated tau^2 value): 0.5041
Test for Residual Heterogeneity:
LRT(df = 1) = 4.9738, p-val = 0.0257
Test of Moderators (coefficients 1:2):
QM(df = 2) = 94.2813, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
xxx$SexFemale 2.9331 0.3022 9.7046 <.0001 2.4359 3.4302 ***
xxx$SexMale 1.1294 0.7376 1.5312 0.1257 -0.0838 2.3425
Test for Selection Model Parameters:
LRT(df = 1) = 22.8488, p-val < .0001
Selection Model Results:
k estimate se zval pval ci.lb ci.ub
0 < p <= 0.025 22 1.0000 --- --- --- --- ---
0.025 < p <= 1 0 0.0000 NA NA NA NA NA