Pybrain Reinforcement Learning Example

Question

As the question states I am looking for a good explanation/example for reinforcement learning in pybrain as the documentation on this confuses me no end, I can get it to work but I don't understand how to apply it to other things.

Thanks Tom

Answer 1

Unfortunately, pybrain's documentation for rl classes is disappointing. I have found this blog quite useful.

---

In summary, you need to identify the following components (for the implementation details follow the tutorial on the link):

1. an environment: env = Environment(...)
2. a task --> task = Task(env)
3. a controller, which is a module (like a table) to keep your action-value information --> controller = Module(...)
4. a learner --> learner = SARSA() --> you may also add an Explorer to the learner. The default is epsilon-greedy with epsilon = 0.3, decay = 0.9999.
5. an agent to integrate controller and learner --> agent = Agent(controller, learner)
6. An experiment to integrate the task and the agent and do actual iterations --> experiment = Experiment(task, agent)

Each of the capitalized classes should be replaced with corresponding class from PyBrain.Then you simply run a do-while cycle to perform the iterations and learn. Note that there are several options to be set by the user, and in real-world problems you most likely need to write sub-classes to generalize the basic classes of pybrain, but the steps will be the same as here.