How to reduce thread switching latency in Python

Question

I have a Python 2.7 app that has 3 producer threads and 1 consumer thread that are connected to a Queue.queue. I'm using get and put, and the producer threads spend most of their time blocked in IO (reading from serial ports) - basically doing nothing. Basically calling serial.read()...

However, I seem to have what I would call a high latency between the time a producer thread puts to the queue and the time the consumer thread gets from the queue, like 25 ms (I'm running a 1 processor Beagle Bone Black (1GHz) on Angstrom Linux).

I would think that if all the processes are blocked, then the elapsed time between put and get should be really small, a few microseconds or so, not tens of milliseconds, except when the consumer thread is actually busy (which is not the case here).

I've read some things online that suggest that Python is guilty of busy spin, and that the GIL in Python is to blame. I guess I would rather not know the reason and just get something that is more responsive. I'm fine with the actual latency of serial transmission (about 1-2 ms).

The code looks basically like

q = Queue.queue

def a1(): 
   while True:
      p = read_serial_packet("/dev/ttyO1")
      p.timestamp = time.time()
      q.put(p)

def a2(): 
   while True:
      p = read_serial_packet("/dev/ttyO2")
      p.timestamp = time.time()
      q.put(p)

def a3(): 
   while True:
      p = read_serial_packet("/dev/ttyO3")
      p.timestamp = time.time()
      q.put(p)

def main():
   while True:
      p = q.get()
      d = time.time() - p.timestamp
      print str(d)

and there are 4 threads running a1, a2,a3 and main.

Here are some sample times

0.0119640827179
0.0178141593933
0.0154139995575
0.0192430019379
0.0185649394989
0.0225830078125
0.018187046051
0.0234098434448
0.0208261013031
0.0254039764404
0.0257620811462

Is this something that is "fixed" in Python 3?

Answer 1

As @fileoffset hinted, the answer seems to be switching from threading (which suffers from the fact that the Python GIL does not actually do "real" threading) to multiprocessing, which has several python processes instead of threads.

The conversion from threading to multiprocessing looks like this:

useMP = True  # or False if you want threading

if useMP:
    import multiprocessing
    import multiprocessing.queues
    import Queue # to import Queue.Empty exception, but don't use Queue.Queue
else:
    import threading
    import Queue

...


    if useMP:
        self.event_queue = multiprocessing.queues.Queue()
        t1 = multiprocessing.Process(target=self.upstream_thread)
        t2 = multiprocessing.Process(target=self.downstream_thread)
        t3 = multiprocessing.Process(target=self.scanner_thread)
    else :
        self.event_queue = Queue.Queue()
        t1 = threading.Thread(target=self.upstream_thread)
        t2 = threading.Thread(target=self.downstream_thread)
        t3 = threading.Thread(target=self.scanner_thread)

The rest of the API looks the same.

There is one other important issue though that was not easy to migrate and is left as an exercise. The issue is catch Unix signals, such as SIGINT or Ctrl-C handlers. Previously, the master thread catches the signal and all the other threads ignore it. Now, the signal is sent to all processes. So you have to be careful about catching KeyboardInterrupt and installing signal handlers. I don't think I did it the right way, so I am not going to elaborate... :)