Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiple ROC curves in one plot ROCR

Tags:

plot

r

roc

Is it possible to plot the roc curve for diffrent classifiers in the same plot using the ROCR package? I've tried:

>plot(perf.neuralNet, colorize=TRUE) >lines(perf.randomForest) 

But I get:

Error en as.double(y) : cannot coerce type 'S4' to vector of type 'double'

Thank you!

like image 656
kahlo Avatar asked Dec 29 '12 19:12

kahlo


People also ask

Can ROC curves be plotted for multiple classes?

In order to extend ROC curve and ROC area to multi-label classification, it is necessary to binarize the output. One ROC curve can be drawn per label, but one can also draw a ROC curve by considering each element of the label indicator matrix as a binary prediction (micro-averaging).

How do you make an AUC ROC plot for a multiclass model?

How do AUC ROC plots work for multiclass models? For multiclass problems, ROC curves can be plotted with the methodology of using one class versus the rest. Use this one-versus-rest for each class and you will have the same number of curves as classes. The AUC score can also be calculated for each class individually.

How are ROC curves plotted?

To plot the ROC curve, we need to calculate the TPR and FPR for many different thresholds (This step is included in all relevant libraries as scikit-learn ). For each threshold, we plot the FPR value in the x-axis and the TPR value in the y-axis. We then join the dots with a line. That's it!


2 Answers

The problem with your lines-approach is that there is no generic S4 lines function for an object of class performance defined in the ROCR package. But you can use the generic plot function as you did with an additional add = TRUE argument. For example this is partly from the example page of ?plot.performance:

library(ROCR) data(ROCR.simple) pred <- prediction( ROCR.simple$predictions, ROCR.simple$labels ) pred2 <- prediction(abs(ROCR.simple$predictions +                          rnorm(length(ROCR.simple$predictions), 0, 0.1)),          ROCR.simple$labels) perf <- performance( pred, "tpr", "fpr" ) perf2 <- performance(pred2, "tpr", "fpr") plot( perf, colorize = TRUE) plot(perf2, add = TRUE, colorize = TRUE) 

OR, you can store all your predictions in a matrix and do all the subsequent steps in one:

preds <- cbind(p1 = ROCR.simple$predictions,                  p2 = abs(ROCR.simple$predictions +                  rnorm(length(ROCR.simple$predictions), 0, 0.1)))  pred.mat <- prediction(preds, labels = matrix(ROCR.simple$labels,                  nrow = length(ROCR.simple$labels), ncol = 2) )  perf.mat <- performance(pred.mat, "tpr", "fpr") plot(perf.mat, colorize = TRUE) 

Btw, if you for some reason really wanted to use lines to plot consecutive ROC curves you would have to do sth. like this:

plot(perf)  lines([email protected][[1]], [email protected][[1]], col = 2) 
like image 195
adibender Avatar answered Sep 23 '22 09:09

adibender


Echoing @adibender, and adding a comment: the example doesn't cover how to set separate colors for each individual curve using the second (plot all at once) approach. In this case, pass col as a list:

library(ROCR) data(ROCR.hiv) x   <- prediction(ROCR.hiv$hiv.nn$predictions, ROCR.hiv$hiv.nn$labels) ROC <- performance(x, "tpr", "fpr") plot(ROC, col = as.list(1:10)) 
like image 26
Scott Kaiser Avatar answered Sep 22 '22 09:09

Scott Kaiser