Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Output confusion matrix in Weka from command line

Tags:

weka

I've saved a random forest model to a file using Weka 3.7.9, and I'm now trying to evaluate it against other (very big) set (on some big machines in Amazon EC2). I'm using the following command line:

> java -server -Xmx60000m -cp weka.jar weka.classifiers.Evaluation 
weka.classifiers.trees.RandomForest -T test.arff -l random-forest.model
-i -no-cv

However, the only output I have is something like this:

=== Error on test data ===

Correctly Classified Instances     3252532               80.0686 %
Incorrectly Classified Instances    809651               19.9314 %
Kappa statistic                          0.2884
Mean absolute error                      0.2539
Root mean squared error                  0.3608
Coverage of cases (0.95 level)          98.7413 %
Total Number of Instances          4062183

Whereas I'm looking besides for something like this:

=== Detailed Accuracy By Class ===

                 TP Rate  FP Rate  Precision  Recall   F-Measure  MCC      ROC Area  PRC Area  Class
                 0.804    0.295    0.731      0.804    0.766      0.512    0.826     0.803     buyer
                 0.705    0.196    0.783      0.705    0.742      0.512    0.826     0.798     non-buyer
Weighted Avg.    0.755    0.245    0.757      0.755    0.754      0.512    0.826     0.801     

=== Confusion Matrix ===

     a     b   <-- classified as
 61728 15004 |     a = buyer
 22662 54066 |     b = non-buyer

Please note that, even if I run the full training method again, like this:

> java -Xmx60000m -cp weka.jar weka.classifiers.Evaluation 
weka.classifiers.trees.RandomForest -t train.arff -T test.arff 
-I 10 -K 0 -S 1 -num-slots 8 -d random-forest.model -i -no-cv

I still doesn't show the confusion matrix for the test-data (only for the trained data).

like image 449
Hugo Sereno Ferreira Avatar asked Mar 21 '13 02:03

Hugo Sereno Ferreira


Video Answer


1 Answers

It works when you omit the -no-cv option.

like image 81
Lars Kotthoff Avatar answered Nov 29 '22 06:11

Lars Kotthoff