Menu
In R caret library, if I got a confusion matrix like this below, if there a way to retrieve the overall accuracy 0.992? I can't get this single value out, since I need to store this value and use it for later processing. Is this possible at all?
Prediction A B C D E
A 1114 2 0 0 0
B 9 745 5 0 0
C 0 6 674 4 0
D 0 0 3 640 0
E 0 0 2 1 718
Overall Statistics
Accuracy : 0.992
95% CI : (0.989, 0.994)
No Information Rate : 0.286
P-Value [Acc > NIR] : <2e-16
Kappa : 0.99
Mcnemar's Test P-Value : NA
Statistics by Class:
Class: A Class: B Class: C Class: D Class: E
Sensitivity 0.992 0.989 0.985 0.992 1.000
Specificity 0.999 0.996 0.997 0.999 0.999
Pos Pred Value 0.998 0.982 0.985 0.995 0.996
Neg Pred Value 0.997 0.997 0.997 0.998 1.000
Prevalence 0.286 0.192 0.174 0.164 0.183
Detection Rate 0.284 0.190 0.172 0.163 0.183
Detection Prevalence 0.284 0.193 0.174 0.164 0.184
Balanced Accuracy 0.996 0.992 0.991 0.996 1.000
202
To calculate accuracy, use the following formula: (TP+TN)/(TP+TN+FP+FN). Misclassification Rate: It tells you what fraction of predictions were incorrect. It is also known as Classification Error. You can calculate it using (FP+FN)/(TP+TN+FP+FN) or (1-Accuracy).
The confusion matrix consists of two types of values: the actual and the predicted ones. Precision: Out of all the positive classes we have predicted correctly, how many are actually positive. Sensitivity / true positive rate / Recall: It measures the proportion of actual positives that are correctly identified.
False positive rate (FPR) is calculated as the number of incorrect positive predictions divided by the total number of negatives. The best false positive rate is 0.0 whereas the worst is 1.0. It can also be calculated as 1 – specificity.
Given a confusion matrix cm, the overall accuracy is obtained by overall.accuracy <- cm$overall['Accuracy']
It's the first time I see the caret package, so how did I know this?
Since you didn't provide an example, I searched for an example code for caret confusion matrices. Here it is (I only added assignment in the last statement):
###################
## 3 class example
confusionMatrix(iris$Species, sample(iris$Species))
newPrior <- c(.05, .8, .15)
names(newPrior) <- levels(iris$Species)
cm <- confusionMatrix(iris$Species, sample(iris$Species))
Now, let's take a look what's in the confusion matrix:
> str(cm)
List of 5
$ positive: NULL
$ table : 'table' int [1:3, 1:3] 13 18 19 20 13 17 17 19 14
..- attr(*, "dimnames")=List of 2
.. ..$ Prediction: chr [1:3] "setosa" "versicolor" "virginica"
.. ..$ Reference : chr [1:3] "setosa" "versicolor" "virginica"
$ overall : Named num [1:7] 0.267 -0.1 0.198 0.345 0.333 ...
..- attr(*, "names")= chr [1:7] "Accuracy" "Kappa" "AccuracyLower" "AccuracyUpper" ...
$ byClass : num [1:3, 1:8] 0.26 0.26 0.28 0.63 0.63 0.64 0.26 0.26 0.28 0.63 ...
..- attr(*, "dimnames")=List of 2
.. ..$ : chr [1:3] "Class: setosa" "Class: versicolor" "Class: virginica"
.. ..$ : chr [1:8] "Sensitivity" "Specificity" "Pos Pred Value" "Neg Pred Value" ...
$ dots : list()
- attr(*, "class")= chr "confusionMatrix"
As you may see, the cm object is a list. We see various "byClass" and "overall" statistics. The overall part is obtained by:
overall <- cm$overall
Which gives us a vector of numbers with string indices:
> overall
Accuracy Kappa AccuracyLower AccuracyUpper AccuracyNull AccuracyPValue McnemarPValue
0.2666667 -0.1000000 0.1978421 0.3449492 0.3333333 0.9674672 0.9547790
Now, extracting the relevant value is as simple as:
> overall.accuracy <- overall['Accuracy']
Summary: str is your friend. Another useful function is attributes -- it returns all the attributes of a given object.
169
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
