Beginner's resources/introductions to classification algorithms [closed]

Question

everybody. I am entirely new to the topic of classification algorithms, and need a few good pointers about where to start some "serious reading". I am right now in the process of finding out, whether machine learning and automated classification algorithms could be a worthwhile thing to add to some application of mine.

I already scanned through "How to Solve It: Modern heuristics" by Z. Michalewicz and D. Fogel (in particular, the chapters about linear classifiers using neuronal networks), and on the practical side, I am currently looking through the WEKA toolkit source code. My next (planned) step would be to dive into the realm of Bayesian classification algorithms.

Unfortunately, I am lacking a serious theoretical foundation in this area (let alone, having used it in any way as of yet), so any hints at where to look next would be appreciated; in particular, a good introduction of available classification algorithms would be helpful. Being more a craftsman and less a theoretician, the more practical, the better...

Hints, anyone?

Answer 1

I've always found Andrew Moore's Tutorials to be very useful. They're grounded in solid statistical theory and will be very useful in understanding papers if you choose to read them in the future. Here's a short description:

These include classification algorithms such as decision trees, neural nets, Bayesian classifiers, Support Vector Machines and cased-based (aka non-parametric) learning. They include regression algorithms such as multivariate polynomial regression, MARS, Locally Weighted Regression, GMDH and neural nets. And they include other data mining operations such as clustering (mixture models, k-means and hierarchical), Bayesian networks and Reinforcement Learning

Answer 2

The answer referring to Andrew Moore's tutorials is a good one. I'd like to augment it, however, by suggesting some reading on the need which drives the creation of many classification systems in the first place: identification of causal relationships. This is relevant to many modeling problems involving statistical inference.

The best current resource I know of for learning about causality and classifier systems (especially Bayesian classifiers) is Judea Pearl's book "Causality: models, reasoning, and inference".