starting multiple threads in python

Question

I want the saveData function from the code below to be executed at the same time. But instead, after the first run of saveData I got the error:

Traceback (most recent call last):
  File "preProcess.py", line 70, in <module>
    run()        
  File "preProcess.py", line 61, in run
    thread.start_new_thread(saveData(slice1, slice2, slice3, dset), ("Thread-" + str(i), 1, ) )
TypeError: first arg must be callable

my code is

#!/usr/bin/env python
import sys

import numpy as np
import h5py
import scipy
from PIL import Image
import timeit
import thread

import matplotlib.pyplot as plt

def saveImage(array, filename):
  fig=plt.figure(figsize=(4,3))
  ax=fig.add_subplot(1,1,1)
  plt.axis('off')
  p = plt.imshow(array)
  p.set_cmap('gray')
  extent = ax.get_window_extent().transformed(fig.dpi_scale_trans.inverted())
  plt.savefig(filename, bbox_inches=extent) 

def saveData(value1, value2, value3, dset):
  filename = "tomo1_" + str(value1) + ".png" 
  data = dset[value1::]
  saveImage(data, filename)
  filename = "tomo2_" + str(value2) + ".png" 
  data = dset[:value2:]
  saveImage(data, filename)
  filename = "tomo3_" + str(value3) + ".png" 
  data = dset[::value3]
  saveImage(data, filename)

def run():

  # Reopen the file and dataset using default properties.
  f = h5py.File(sys.argv[1])
  dset = f[sys.argv[2]]

  dim1 = len(dset)
  dim2 = len(dset[0])
  dim3 = len(dset[0][0])

  slice1 = 0
  slice2 = 0
  slice3 = 0
  factor1 = dim1/48
  factor2 = dim2/48
  factor3 = dim3/48
  tic=timeit.default_timer()
  for i in range(0,48):
    thread.start_new_thread(saveData(slice1, slice2, slice3, dset), ("Thread-" + str(i),     1, ) )
    slice1 = slice1 + factor1
    slice2 = slice2 + factor2
    slice3 = slice3 + factor3

  toc=timeit.default_timer()
  print "elapsed time: " + str(toc - tic)

if __name__ == "__main__":
    run()        

I would like to execute the saveData function 48 times at the same time. What I am doing wrong?

Answer 1

As Gryphius pointed out: You mustn't execute saveData in start_new_thread but rather use your function as an ordinary argument to thread.start_new_thread!

The reason is, that thread.start_new_thread will execute the function for you in a Thread. So you have to pass not only the function as an argument (without executing it), but also the arguments of your function:

thread.start_new_thread(saveData,(slice1,slice2,slice3,dset),("Thread-"+str(i),1,))

As you see, the arguments to your function are passed as a tuple.

As a side-note: This idiom, of not calling a function, but passing it along with it's argument is also seen in event-handling functions: callbacks.

Also consider using the Threading package instead of thread it's much more sophisticated and allows you more freedom and functionality!

Answer 2

carefully read the api docs for thread.start_new_thread. It expects the first argument to be a callable, so just saveData in your case instead of saveData(slice1, slice2, slice3, dset). The second argument is a tuple with your function arguments.

try:

thread.start_new_thread(saveData,(slice1, slice2, slice3, dset))