Executing tasks in parallel in python

Question

I am using python 2.7, I have some code that looks like this:

task1() task2() task3() dependent1()  task4() task5() task6() dependent2()  dependent3() 

The only dependencies here are as follows: dependent1 needs to wait for tasks1-3, dependent2 needs to wait for tasks 4-6 and dependent3 needs to wait for dependents1-2... The following would be okay: running the whole 6 tasks first in parallel, then the first two dependents in parallel.. then the final dependent

I prefer to have as much tasks as possible running in parallel, I've googled for some modules but I was hoping to avoid external libraries, and not sure how the Queue-Thread technique can solve my problem (maybe someone can recommend a good resource?)

Answer 1

The builtin threading.Thread class offers all you need: start to start a new thread and join to wait for the end of a thread.

import threading  def task1():     pass def task2():     pass def task3():     pass def task4():     pass def task5():     pass def task6():     pass  def dep1():     t1 = threading.Thread(target=task1)     t2 = threading.Thread(target=task2)     t3 = threading.Thread(target=task3)      t1.start()     t2.start()     t3.start()      t1.join()     t2.join()     t3.join()  def  dep2():     t4 = threading.Thread(target=task4)     t5 = threading.Thread(target=task5)      t4.start()     t5.start()      t4.join()     t5.join()  def dep3():     d1 = threading.Thread(target=dep1)     d2 = threading.Thread(target=dep2)      d1.start()     d2.start()      d1.join()     d2.join()  d3 = threading.Thread(target=dep3) d3.start() d3.join() 

Alternatively to join you can use Queue.join to wait for the threads end.