What are the other forms of evaluation besides a confusion matrix?

Question

Homework

I need an other form of evaluation besides a confusion matrix to visualize the performance of an algorithm.

Answer 1

Precision vs. Recall curves are also good (also here).

If it's something like a regression, many stats folks advise plotting it as a fitted curve over top of the data, or perhaps many different two-dimensional projections of the fitted curve if it is high-dimensional. Also advisable is to weight the confidence intervals and things so that they show up fainter than other parts of the plot, to visually down-weight things according to how improbable they are. Here's a recent post from Andrew Gelman on that: (link)

If you are making a generative model, like computing a Bayesian posterior distribution, then posterior predictive checking is a good one. If you're doing regressions, don't underestimate the value of a table that summarizes the coefficients and the R^{2} value.

If your data naturally fits over top of some domain, like a part of the x-y plane, or you have data for each state in the US, etc., then trying to overlay your data above the domain is always good. Make one plot showing the distribution of the data (such as a histogram, but generally much better if it's a density over top of some visually-meaningful part of the problem domain), then replicate the same exact plot with the output of the algorithm. Visually, severe differences between your data's distribution and the output of the algorithm will draw attention to places where the algorithm does not lead to correct inference.

That's really the key. The goal is valid inference, not hitting a relatively low amount of classification errors, etc. If you get the nice classification error, but your method is overfitting, it's probably going to show up in some of these types of plots.