Finding an optimum learning rule for an ANN

Question

How do you find an optimum learning rule for a given problem, say a multiple category classification?

I was thinking of using Genetic Algorithms, but I know there are issues surrounding performance. I am looking for real world examples where you have not used the textbook learning rules, and how you found those learning rules.

Answer 1

Nice question BTW.

classification algorithms can be classified using many Characteristics like:

1. What does the algorithm strongly prefer (or what type of data that is most suitable for this algorithm).
2. training overhead. (does it take a lot of time to be trained)
3. When is it effective. ( large data - medium data - small amount of data ).
4. the complexity of analyses it can deliver.

Therefore, for your problem classifying multiple categories I will use Online Logistic Regression (FROM SGD) because it's perfect with small to medium data size (less than tens of millions of training examples) and it's really fast.

Another Example:

let's say that you have to classify a large amount of text data. then Naive Bayes is your baby. because it strongly prefers text analysis. even that SVM and SGD are faster, and as I experienced easier to train. but these rules "SVM and SGD" can be applied when the data size is considered as medium or small and not large.

In general any data mining person will ask him self the four afomentioned points when he wants to start any ML or Simple mining project.

After that you have to measure its AUC, or any relevant, to see what have you done. because you might use more than just one classifier in one project. or sometimes when you think that you have found your perfect classifier, the results appear to be not good using some measurement techniques. so you'll start to check your questions again to find where you went wrong.

Hope that I helped.