I have the same problem of a prevous request HOW to use the survivalROC (or another library in R) to get optimal cut-off values? I want to use the time-dependent survivalROC package.according to the,reference material,it only gives a set of ordered cut-off values .eg. -------------------------------------------------------------------------------- data(mayo) str(mayo) attach(mayo) ROC.1=survivalROC(Stime=time,status=censor,marker=mayoscore4,predict.time=365,lambda=0.05) str(ROC.1) plot(ROC.1$FP, ROC.1$TP, type="l", xlim=c(0,1), ylim=c(0,1), xlab=paste( "FP", "\n", "AUC = ",round(ROC.1$AUC,3)), ylab="TP",main="Mayoscore 4, Method = NNE \n Year = 1") abline(0,1) List of 6 $ cut.values : num [1:313] -Inf 4.58 4.9 4.93 4.93 ... *only 5 values * $ TP : num [1:313] 1 0.999 0.999 0.999 0.998 ... $ FP : num [1:313] 1 0.997 0.993 0.99 0.987 ... $ predict.time: num 365 $ Survival : num 0.93 $ AUC : num 0.888 -------------------------------------------------------------------------------- so i dont know how to use the survivalROC to get optimal cut-off values?(only 5 values) Thank you very much! Mario Petretta Dipartimento di Medicina Clinica Scienze Cardiovascolari e Immunologiche Facolt? di Medicina e Chirurgia Universit? di Napoli Federico II 081 - 7462233
HOW to use the survivalROC to get optimal cut-off values?
3 messages · Prof. Mario Petretta, David Winsemius
On Dec 5, 2010, at 11:14 AM, petretta at unina.it wrote:
I have the same problem of a prevous request HOW to use the survivalROC (or another library in R) to get optimal cut-off values? I want to use the time-dependent survivalROC package.according to the,reference material,it only gives a set of ordered cut-off values .eg.
Optimality specification requires some sort of valuation of incorrect decisions. If you are willing to defend a choice that a false positive has exactly the same loss as a false negative, which is generally not the case in medical decision-making, then the point on the ROC curve which is closest to the upper left-hand corner is "optimal". Having only 5 values is getting pretty close to violating the presumption of ROC analysis that the result be at least pseudo- continuous. I have see quite a few "ROC curves in this situation that do not have a clear winner ( now assuming equal cost for FP and FN which I already said was usually a faulty assumption) because the closest point on the curve was in the middle of the line segment between two of the points. I'm not sure that the typical practice of plotting ROC curves with slanting line segments is valid. There is no information between those discrete points. You should probably be using a table rather than a curve in this situation.
-------------------------------------------------------------------------------- data(mayo) str(mayo) attach(mayo) ROC. 1 = survivalROC (Stime =time,status=censor,marker=mayoscore4,predict.time=365,lambda=0.05) str(ROC.1) plot(ROC.1$FP, ROC.1$TP, type="l", xlim=c(0,1), ylim=c(0,1), xlab=paste( "FP", "\n", "AUC = ",round(ROC.1$AUC,3)), ylab="TP",main="Mayoscore 4, Method = NNE \n Year = 1") abline(0,1) List of 6 $ cut.values : num [1:313] -Inf 4.58 4.9 4.93 4.93 ... *only 5 values * $ TP : num [1:313] 1 0.999 0.999 0.999 0.998 ... $ FP : num [1:313] 1 0.997 0.993 0.99 0.987 ... $ predict.time: num 365 $ Survival : num 0.93 $ AUC : num 0.888 -------------------------------------------------------------------------------- so i dont know how to use the survivalROC to get optimal cut-off values?(only 5 values) Thank you very much! Mario Petretta Dipartimento di Medicina Clinica Scienze Cardiovascolari e Immunologiche Facolt? di Medicina e Chirurgia Universit? di Napoli Federico II 081 - 7462233
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
David Winsemius, MD West Hartford, CT
On Dec 5, 2010, at 11:33 AM, David Winsemius wrote:
On Dec 5, 2010, at 11:14 AM, petretta at unina.it wrote:
I have the same problem of a prevous request HOW to use the survivalROC (or another library in R) to get optimal cut-off values? I want to use the time-dependent survivalROC package.according to the,reference material,it only gives a set of ordered cut-off values .eg.
Optimality specification requires some sort of valuation of incorrect decisions. If you are willing to defend a choice that a false positive has exactly the same loss as a false negative, which is generally not the case in medical decision-making, then the point on the ROC curve which is closest to the upper left-hand corner is "optimal".
Snipped comments about small nubers of discrete values as that was a misunderstanding on my part.
-------------------------------------------------------------------------------- data(mayo) str(mayo) attach(mayo) ROC. 1 = survivalROC (Stime =time,status=censor,marker=mayoscore4,predict.time=365,lambda=0.05) str(ROC.1) plot(ROC.1$FP, ROC.1$TP, type="l", xlim=c(0,1), ylim=c(0,1), xlab=paste( "FP", "\n", "AUC = ",round(ROC.1$AUC,3)), ylab="TP",main="Mayoscore 4, Method = NNE \n Year = 1") abline(0,1) List of 6 $ cut.values : num [1:313] -Inf 4.58 4.9 4.93 4.93 ... *only 5 values * $ TP : num [1:313] 1 0.999 0.999 0.999 0.998 ... $ FP : num [1:313] 1 0.997 0.993 0.99 0.987 ... $ predict.time: num 365 $ Survival : num 0.93 $ AUC : num 0.888 -------------------------------------------------------------------------------- so i dont know how to use the survivalROC to get optimal cut-off values?(only 5 values)
I really do not understand what you are asking when you ask about "5
values". (I also see that another student GY QIAN in SHANGHAI,CHINA
asked almost the same odd question on rhelp about 6 weeks ago making
me wonder if there is some online class you are both taking for which
this is homework?)
There are 312 finite values for each of cut.values, FP, and TP. Is
someone, your course instructor or collaborator, asking for a
comparison at 5 selected cutpoints? I generally use Hmisc::describe
for quick looks:
> describe(ROC.1$cut.values[-1])
ROC.1$cut.values[-1]
n missing unique Mean .05 .10 .25 .50 .
75 .90 .95
312 0 312 6.538 5.320 5.459 5.842 6.312
6.949 7.966 8.675
lowest : 4.581 4.900 4.926 4.932 4.946, highest: 9.510 9.568
10.185 10.479 10.629
You could programmatically ask what values of the index (1:313)
minimizes the sum of FN and (1-TP) = FP. And then you can use to index
the TP and "FP" values. I guess it is common practice to call 1-
specificity the "false positive rate" but I for one find that very
confusing. even misleading, since it's not clear from the term what
the denominator for such a "rate" really should be. (It's also not
really a rate since no time is involved in the calculation.) At any
rate, as it were:
> with(ROC.1, which.min(1-TP+ FP))
[1] 259
> with(ROC.1, points(FP[259], TP[259], cex=3, col="red" ))
David Winsemius, MD West Hartford, CT