mpi4py Pickling error when trying to send classes

Question

im using mpi4py to split some calculations among several procs. Basically Im just calculating the volumes of some convex hull, which I create using tvtk and mayavi.

only the first proc imports These libs:

...
if rank==0:
   from tvtk.api import tvtk
   from mayavi Import mlab
   ...
   mlab.figure(size=(1024,768),bgcolor=(1,1,1))
   ...

Then, I try to share the objects mlab and tvtk among all procs:

for i in range(comm.Get_size()):
    comm.send(mlab,dest=i)
    comm.send(tvtk,dest=i)
....

The following steps would look something like...

Points=local_data
ug=tvtk.UnstructuredGrid(Points=Points)
...
dataname="Data %s " % rank
ds=mlab.pipeline.add_dataset(ug,name=dataname)
delaunay=mlab.pipeline.delaunay3d(ds,name=dataname)
... calc volume...

However, obviously its not possible to send instances/classes (or whatever mlab and tvtk are), as I always get the following error:

comm.send(mlab,dest=i)
File "Comm.pyx", line 753, in mpi4py.MPI.Comm.send (src/mpi4py.MPI.c:53848)
File "pickled.pxi", line 122, in mpi4py.MPI.PyMPI_send (src/mpi4py.MPI.c:20409)
 File "pickled.pxi", line 39, in mpi4py.MPI._p_Pickle.dump (src/mpi4py.MPI.c:19503)
cPickle.PicklingError: Can't pickle <type 'module'>: attribute lookup __builtin__.module failed

Is there any way to 'share' the instances mlab and tvtk among all procs?

EDIT: short example; can u get this working?

from mpi4py import MPI
comm=MPI.COMM_WORLD

size=comm.Get_size()
rank=comm.Get_rank()

if rank==0:
        from tvtk.api import tvtk
        from mayavi import mlab

if __name__=='__main__':
        if rank==0:
                for i in range(size):
                        comm.send(mlab,dest=i)
                        comm.send(tvtk,dest=i)
        else:
                local_mlab=comm.recv(mlab,source=0)
                local_tvtk=comm.recv(tvtk,source=0)

Answer 1

I guess the following might work in your case

from mpi4py import MPI
MPI._p_pickle.dumps = dill.dumps
MPI._p_pickle.loads = dill.loads