Skip to content

Jagged ROC curves?

3 messages · Brian Smith, Marc Schwartz

Brian Smith
# 
Hi,

I was trying to draw some ROC curves (prediction of case/control status),
but seem to be getting a somewhat jagged plot. Can I do something that
would 'smooth' it somewhat? Most roc curves seem to have many incremental
changes (in x and y directions), but my plot only has 4 or 5 steps even
though there are 22 data points. Should I be doing something differently?

How can I provide a URL/attachment for my plot? Not sure if I can provide
reproducible code, but here is some pseudocode, let me know if you'd like
more details:

#####
## generate roc and auc values
#####
library(pROC)
library(AUCRF)

getROC <- function(d1train,d1test){
my_model <- AUCRF(formula= status ~ ., data=d1train,
ranking='MDA',ntree=1000,pdel=0.05)
  my_opt_model <- my_model$RFopt

  my_probs <- predict(my_opt_model, d1test, type = 'prob')
  my_roc <- roc(d1test[,resp_col] ~ my_probs[,2])
  aucval <- round(as.numeric(my_roc$auc),4)
return(my_roc)
}


roc_1 <- getROC(dat1,dat1test)
plot.roc(roc_1,col="brown3")
Call:
roc.formula(formula = d1test[, resp_col] ~ ibd_probs[, 2])

Data: ibd_probs[, 2] in 3 controls (d1test[, resp_col] 0) < 19 cases
(d1test[, resp_col] 1).
Area under the curve: 0.8596
[1] 1.00000000 0.94736842 0.94736842 0.94736842 0.89473684 0.84210526
0.78947368 0.73684211 0.68421053 0.68421053
[11] 0.63157895 0.57894737 0.52631579 0.47368421 0.42105263 0.36842105
0.31578947 0.26315789 0.21052632 0.15789474
[21] 0.10526316 0.05263158 0.00000000
[1] 0.0000000 0.0000000 0.3333333 0.6666667 0.6666667 0.6666667 0.6666667
0.6666667 0.6666667 1.0000000 1.0000000
[12] 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000
1.0000000 1.0000000 1.0000000 1.0000000
[23] 1.0000000


many thanks!
Marc Schwartz
# 
ROC curves are typically step functions of some nature, depending upon your thresholds, so the default behavior is not going to be smoothed.

I am not sure how they (AUCRF and pROC) may interact, but look at the ?smooth function in the latter package to see if it might help.

To your second point, if your plot is a png/jpg file, you could attach it to your post here, if that was your desire. Otherwise, you could post it to a cloud based repository, like Dropbox, and provide the URL for public sharing here. The R lists support limited binary attachment types and png/jpg/pdf/ps are supported.

Regards,

Marc Schwartz
Brian Smith
# 
Hi Marc,

I tried to attache the png file of the plot, but the mailing list blocked
it!


"For the attached two png files (test_roc.png & test_roc_smooth.png)

1. Using 'plot' function:

plot(c(1,0),c(0,1), type='l', lty=3, xlim=c(1.01,-0.01),
ylim=c(-0.01,1.01), xaxs='i', yaxs='i', ylab='', xlab='')
plot(roc_1,col="brown3", lwd=2, add=T, lty=1)

2. Using the 'smooth' function:

plot(c(1,0),c(0,1), type='l', lty=3, xlim=c(1.01,-0.01),
ylim=c(-0.01,1.01), xaxs='i', yaxs='i', ylab='', xlab='')
plot(smooth(roc_1),col="brown3", lwd=2, add=T, lty=1)


I guess most ROCs that I've seen are somewhere in between, i.e. they have a
little jaggedness, but not as much as in plot #1 above"


thanks!

On Mon, Jun 26, 2017 at 12:59 PM, Marc Schwartz <marc_schwartz at me.com>
wrote: