Can evolutionary computation be a method of reinforcement learning?

Question

What is evolutionary computation? Is it a method of reinforcement learning? Or a separate method of machine learning? Or maybe none?

Please, cite references used to answer this question.

Answer 1

There are evolutionary methods that are explicitly aimed at solving the reinforcement learning problem. The subfield typically goes by the name of Learning Classifier Systems (LCS) or occasionally Genetics-Based Machine Learning (GBML).

Aside from that, I'm not sure your question has a very well-defined answer. It basically boils down to "what is machine learning?" There's no canon that we've all agreed on for how to answer that question. For some, EC might be a part of that subfield. For others, it isn't. I just sampled a handful of ML textbooks from my shelf, and about half contained material on evolutionary methods. I suspect 15 years ago that fraction would have been higher, but fashions change, and machine learning is very nearly a subfield of statistics now. EC methods don't fit that mold very well.

Answer 2

Evolutionary computation, or evolutionary algorithms, are optimization algorithms, which, when applied to a neural network (as in neuro-evolution) can certainly be classified as a form of reinforcement learning, although it works a bit different than the usual reinforcement learning algorithm.

Generally, in evolutionary algorithms such as genetic algorithms, or evolution strategy, you have a whole population of individuals to be optimized. For each of those individuals, a quality function is used to determine their 'fitness' (as in 'survival of the fittest'), and the best individuals are selected for the next generation. Those 'parents' are then randomly duplicated, modified, mutated, or even recombined with each others -- how exactly this is done is a bit different in each of the different algorithms. Finally, those new mutated and/or recombined parents form the population for the next generation, and the process starts again, until some desired quality is reached, or the quality levels out.

In the case of neuro-evolution, the individuals are neural networks, which are mutated by randomly changing weights (whereas in classical neural networks the weights are updated according to very precise mathematical rules) or even altering their topology, and the quality of the individuals is determined by how well they perform on the training data.

Sorry, no hard scientific reference here, but maybe this still helped clearing things up a bit.