How can I generate a plot of positive predictive value (PPV) vs various cut-off points for classifications?

Question

I have generated some scores to help predict whether or not something is yes (1) or no (0), let's say the data consists of:

scores = c(10:20)

response = c(0,0,1,0,1,0,1,1,0,1,1)

mydata = data.frame(scores, response)

I can do an ROC analysis, which gives an AUC of .77:

roc(response = mydata$response, predictor = mydata$scores) 

Now, how exactly do I see what happens when various cut-offs are chosen? I'd like to have cut-offs on the x-axis (let's say 13,14,15,16,17) and PPV on the y-axis. What's a nice way of doing this? What functions/packages do I need?

Answer 1

I will give an answer based around the pROC package*. It is possible to obtain similar results using the ROCR package as well.

You want to use the coords function, which can compute several common statistics at some given thresholds. For instance, in order to get the PPV at all thresholds, you can do the following:

library(pROC)
r <- roc(response = response, predictor = scores)
coordinates <- coords(r, x = "all", input = "threshold", ret = c("threshold", "ppv"))

You can then plot those values:

plot(t(coordinates))

Replace "all" by the thresholds of interest:

 coordinates <- coords(r, x = c(13, 14, 15, 16, 17), input = "threshold", ret = c("threshold", "ppv"))

_{* Disclaimer: I am the author of the pROC package.}