interpreting bootstrap corrected slope [rms package]
Adam - the very low amount of optimism suggests that you have a large sample size and that your model was completely pre-specified. If you did any feature/variable selection or made any model changes in a way that was not blinded to Y then you are not using the software correctly. But you are right the slope decrement indicates a bit of overfitting on an absolute calibration scale. The harm done by this can be partially interpreted by the Emax value of 0.05 indicated the maximum absolute calibration error is estimated to be 0.05 on the probability scale. If your exceedence probabilities for the middle Y category have a wide range then 0.05 isn't so bad. Frank
apeer wrote:
Dear List:
Below is the validation output of a fitted ordinal logistic model
using the bootstrap in the rms package. My interpretation is that
most of the corrected indices indicate little overfitting, however the
slope seems to indicate that the model is too optimistic. Given that
most of the corrected indices seem reasonable, would it be appropriate
to use this model on future data if the corrected intercept and slope
estimates are used?
index.orig training test optimism index.corrected n
Dxy 0.9932 0.9940 0.9905 0.0035 0.9897 363
R2 0.9291 0.9364 0.9163 0.0202 0.9089 363
Intercept 0.0000 0.0000 0.0233 -0.0233 0.0233 363
Slope 1.0000 1.0000 0.7836 0.2164 0.7836 363
Emax 0.0000 0.0000 0.0582 0.0582 0.0582 363
D 0.9118 0.9190 0.8915 0.0275 0.8844 363
U -0.0110 -0.0110 0.0124 -0.0234 0.0124 363
Q 0.9228 0.9299 0.8791 0.0508 0.8720 363
B 0.0205 0.0172 0.0239 -0.0067 0.0272 363
Any input is much appreciated.
Thanks,
Adam
______________________________________________ R-help@ mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
----- Frank Harrell Department of Biostatistics, Vanderbilt University -- View this message in context: http://r.789695.n4.nabble.com/interpreting-bootstrap-corrected-slope-rms-package-tp3928314p3928467.html Sent from the R help mailing list archive at Nabble.com.