Tkinter: Wait for item in queue

Question

I’m using a queue to exchange messages between a background thread and a Tk GUI application. Currently, this is done by calling a query method every now and then.

def read_queue(self):
    try:
        self.process(self.queue.get(False)) # non-blocking
    except Queue.Empty:
        pass
    finally:
        self.after(UPDATE_TIME, self.read_queue)

The problem with this approach is that if UPDATE_TIME is too large, the application will process new items slower than possible. If it is too small, Tk spends most of the time checking the queue although it could do other stuff in the meantime.

Is there a way to automatically trigger the read_queue method whenever a new item arrives in the queue? (I could certainly call a method on Tk when the background thread fills the queue but I fear that this gives me some concurrency issues – that’s why I’m using queues after all.)

Answer 1

One option might be mtTkinter http://tkinter.unpythonic.net/wiki/mtTkinter

Here is another example of using event_generate from a background thread:

##The only secure way I found to make Tkinter mix with threads is to never  
##issue commands altering the graphical state of the application in another  
##thread than the one where the mainloop was started. Not doing that often  
##leads to random behaviour such as the one you have here. Fortunately, one  
##of the commands that seems to work in secondary threads is event_generate,  
##giving you a means to communicate between threads. If you have to pass  
##information from one thread to another, you can use a Queue.
##
##This obviously complicates things a bit, but it may work far better.  
##Please note that the 'when' option *must* be specified in the call to  
##event_generate and *must not* be 'now'. If it's not specified or if it's  
##'now', Tkinter may directly execute the binding in the secondary thread's  
##context. (Eric Brunel)

import threading
import time
import Queue
from Tkinter import *

## Create main window
root = Tk()

## Communication queue
commQueue = Queue.Queue()

## Function run in thread
def timeThread():
    curTime = 0
    while 1:
        ## Each time the time increases, put the new value in the queue...
        commQueue.put(curTime)
        ## ... and generate a custom event on the main window
        try:
            root.event_generate('<<TimeChanged>>', when='tail')
        ## If it failed, the window has been destoyed: over
        except TclError:
            break
        ## Next
        time.sleep(1)
        curTime += 1

## In the main thread, do usual stuff
timeVar = IntVar()
Label(root, textvariable=timeVar, width=8).pack()

## Use a binding on the custom event to get the new time value
## and change the variable to update the display
def timeChanged(event):
    timeVar.set(commQueue.get())

root.bind('<<TimeChanged>>', timeChanged)

## Run the thread and the GUI main loop
th=threading.Thread(target=timeThread)
th.start()

root.mainloop()

There is also mention of using after_idle in a similar way.
ie. root.after_idle(timeChanged)

Answer 2

SUMMARY: I wouldn't use "noob oddy's example code" -- is a fundamentally flawed approach.

I'm not a python guru, but the example code provided by "noob oddy" (which calls root.event_generate(...) within the background thread) appears to be a "fundamentally flawed approach". i.e., there are several articles on the internet which state "to never invoke Tkinter functions/object methods outside the context of the 'GUI thread'" (which is typically the main thread). His example works "most of the time", but if you increase the event generation rate, then the example's "crash rate" will increase -- however, specific behavior is dependent on the event generation rate and the platform's performance characteristics.

For example, using his code with Python 2.7.3, if you change:

       time.sleep(1)

to:

       time.sleep(0.01)

then the script/app will typically crash after 'x' number of iterations.

After much searching, if you "must use Tkinter", then it appears the most "bullet proof method" of getting information from a background thread to the GUI thread is to use 'after()' widget method to poll a thread-safe object (such as 'Queue'). e.g.,

################################################################################
import threading
import time
import Queue
import Tkinter      as Tk
import Tkconstants  as TkConst
from ScrolledText import ScrolledText
from tkFont       import Font

global top
global dataQ
global scrText

def thread_proc():
    x = -1
    dataQ.put(x)
    x = 0
    for i in xrange(5):
        for j in xrange(20):
            dataQ.put(x)
            time.sleep(0.1)
            x += 1
        time.sleep(0.5)
    dataQ.put(x)

def on_after_elapsed():
    while True:
        try:
            v = dataQ.get(timeout=0.1)
        except:
            break
        scrText.insert(TkConst.END, "value=%d\n" % v)
        scrText.see(TkConst.END)
        scrText.update()
    top.after(100, on_after_elapsed)

top     = Tk.Tk()
dataQ   = Queue.Queue(maxsize=0)
f       = Font(family='Courier New', size=12)
scrText = ScrolledText(master=top, height=20, width=120, font=f)
scrText.pack(fill=TkConst.BOTH, side=TkConst.LEFT, padx=15, pady=15, expand=True)
th = threading.Thread(target=thread_proc)
th.start()
top.after(100, on_after_elapsed)
top.mainloop()
th.join()
## end of file #################################################################

Answer 3

Polling can be eliminated from the Ken Mumme solution by using os.pipe to synchronise between the two threads.

tkinter has a createFilehandler method that can be used to add a file descriptor into tk's select loop. You can then signal that something is ready in the queue by writing a byte into the pipe.

The solution looks like this:

import Queue
import os

uiThreadQueue = Queue.Queue() ;

pipe_read, pipe_write = os.pipe() ;

# call one function from the queue.  Triggered by the 
# pipe becoming readable through root.tk.createfilehandler().
def serviceQueue(file, mask):
    os.read(pipe_read, 1) 
    func = uiThreadQueue.get() ;
    func() 

# enqueue a function to be run in the tkinter UI thread.
# best used as inUIThread(lambda: self.callSomeFunction(arg1,arg2,arg3))
def inUIThread(f):
    uiThreadQueue.put(f)
    os.write(pipe_write, "x")

... set up your widgets, start your threads, etc.....


root.tk.createfilehandler(pipe_read, tkinter.READABLE, serviceQueue)
root.mainloop()

I'm not a python expert; apologies if I've messed up on any coding conventions. I'm great with pipes, though :)