How to split python tasks across a network

Question

I have a python program that performs several independent and time consuming processes. The python code is generally an automater, that calls into several batch files via popen.

The program currently takes several hours, so I'd like to split it up across multiple machines. How can I split tasks to process in parallel with python, over an intranet network?

Answer 1

There are many Python parallelisation frameworks out there. Just two of the options:

• The parallel computing facilities of IPython

• The parallelisation framework jug