C++ Reinforcement learning and smart pointers

Question

I am doing my Masters project on robotic's sensorimotor online learning using reinforcement learning methods (Q,sarsa,TD(λ),Actor-Critic,R,etc). I am currently designing the framework on which both higher level reinforcement learning and lower level robot API control will be using.

Since the states are robot sensor dependant and may (will) increase exponentially, I will be allocating them on the heap. Since this can create alot of problems, bugs, etc, and since parallelization (i.e. threading) is an aspect of reinforcement learning I want to explore, I am not yet sure of what kind of smart pointers to use.

Designing my own template/class for a smart pointer will take time and debugging, which I do not have. So, I am wondering, should I use STL's auto_ptr? I see they have issues being used in vectors. Should I use boost::shared_ptr? The states will have to be shared among many classes and algorithms. Or should I use boost::ptr_vector? Since the states will reside in a task container class in a vector, would this be sufficient? The states will have to be shared, copyable, referencable, serializable, non constant, thread-safe and will not be deleted. Also, memory space and computation time are important.

What do you recommend as the best smart ptr implementation for such a task ?

Thank you!

---

It seems like I will have to try using boost::ptr_vector with class State, and if this proves unefficient, then use std::vector < std::unique_ptr > and enable 0X. Thank you all for your answers and suggestions !

Answer 1

Single-ownership pointers are the harder to misuse, modulo the design of std::auto_ptr. You could consider using boost::scoped_ptr, which is safer yet (it can't transfer ownership though and can't be returned from a function). When it comes to containers, you could use a pointer container, but it's also fine to use e.g. std::vector without smart pointers if the types involved are not used polymorphically.

Shared ownership should remain an exceptional case; don't overuse boost::shared_ptr.

Answer 2

Other suggestions missed one. boost::intrusive_ptr performs better than shared_ptr because it doesn't need a second allocation to hold the reference count. The downside of intrusive_ptr is a tiny bit of extra bookkeeping and no ability to use weak_ptr.

Answer 3

I never found a smart ptr class that I liked when I was doing c++. I wrote my own in the end.

It had a cache feature for speed so it you kept allocating and freeing memory it could just hang on to the memory it had and reuse it. Also it had no default constructor so you had to pass it to functions/methods by reference else the compiler would show an error, this was so it would not create a temp copy of memory especially when dealing with large image files.

It would not take too long to write your own code and you can add your own bounds checking code to it as well.