How do I change the serializer that my multiprocessing.mangers.BaseManager subclass uses to cPickle?

Question

Using Python 2.7,

I am passing many large objects across processes using a manager derived from multiprocessing.managers. BaseManager and I would like to use cPickle as the serializer to save time; how can this be done? I see that the BaseManager initializer takes a serializer argument, but the only options appear to be pickle and xmlrpclib.

Answer 1

It seems like you can't strictly do what your asking.

In fact, there's a fork of multiprocessing, pathos, written by the creators of an alternative to pickle, dill - also because of the limited ability to control the serializer.

I would personally suggest you use ipython.parallel, as it seems more actively maintained.

See more details on this matter in this piece Parallelism and Serialization.