wxPython threads blocking

Question

This is in the Phoenix fork of wxPython.

I'm trying to run a couple threads in the interests of not blocking the GUI.

Two of my threads work fine, but the other one never seem to hit its bound result function. I can tell that it's running, it just doesn't seem to properly post the event.

Here's the result function for the main calculation threads:

def on_status_result(self, event):
    if not self.panel.progress_bar.GetRange():
        self.panel.progress_bar.SetRange(event.data.parcel_count)
    self.panel.progress_bar.SetValue(event.data.current_parcel)
    self.panel.status_label.SetLabel(event.data.message)

Here's how I'm binding them:

from wx.lib.pubsub.core import Publisher
PUB = Publisher()

Here's how I'm binding the method:

def post_event(message, data):
    wx.CallAfter(lambda *a: Publisher().sendMessage(message, data=data))

And here are the threads. The first one does not work, but the second two do:

class PrepareThread(threading.Thread):
    def __init__(self, notify_window):
        threading.Thread.__init__(self)
        self._notify_window = notify_window
        self._want_abort = False

    def run(self):
        while not self._want_abort:
            for status in prepare_collection(DATABASE, self._previous_id, self._current_id, self._year, self._col_type,
                                             self._lock):
                post_event('prepare.running', status)
        post_event('prepare.complete', None)
        return None

    def abort(self):
        self._want_abort = True


class SetupThread(threading.Thread):
    def __init__(self, notify_window):
        threading.Thread.__init__(self)
        self._notify_window = notify_window
        self._want_abort = False

    def run(self):
        while not self._want_abort:
            do_more_stuff_with_the_database()
            return None

    def abort(self):
        self._want_abort = True


class LatestCollectionsThread(threading.Thread):
    def __init__(self, notify_window):
        threading.Thread.__init__(self)
        self._notify_window = notify_window
        self._want_abort = False

    def run(self):
        while not self._want_abort:
            do_stuff_with_my_database()
            return None

    def abort(self):
        self._want_abort = True

prepare_collection is a function that yields Status objects that looks like this:

class Status:
    def __init__(self, parcel_count, current_parcel, total, message):
        self.parcel_count = parcel_count
        self.current_parcel = current_parcel
        self.total = total
        self.message = message

Here's how I'm creating/starting/subscribing the PrepareThread:

MainForm(wx.Form):
    prepare_thread = PrepareThread(self)
    prepare_thread.start()

    self.pub = Publisher()
    self.pub.subscribe(self.on_status_result, 'prepare.running')
    self.pub.subscribe(self.on_status_result, 'prepare.complete')

    def on_status_result(self, event):
        if not self.panel.progress_bar.GetRange():
            self.panel.progress_bar.SetRange(event.data.parcel_count)
        self.panel.progress_bar.SetValue(event.data.current_parcel)
        self.panel.status_label.SetLabel(event.data.message)

I've tried stubbing out prepare_collection with range(10), but I still don't ever hit the event handler.

Answer 1

This could be quite an involved answer and it's a little bit hard to work out exactly which of your snippets you have in each part of your code (i.e. which files they all live in). I am assuming that you want to keep the pubsub method of doing this (which I think is a good idea). If the structure of your real project is very complex, you might need more complex management of the Publisher than I use here - let me know...

---

Here goes - I'll put the spoiler first - here's a one file solution for what you seem to want - a panel with a button to kick off the prepare thread, status bar and handler for when preparation has finished. Tested with wxPython Phoenix 64 bit Python 3 and old fashioned wxPython on Python 2.7. Both on Windows - but I can spin it up in a Linux box if necessary.

Summarising the important (non boiler-plate) bits of that file

You need a single Publisher object that your threads send messages to and your main thread (MainForm in your example I guess) subscribes to. You could manage a Publisher for each thread, but I think here you only need one for the PrepareThread, so I'm going to go with that model for now.

At the top of your file, use

from wx.lib.pubsub import pub

This lets pubsub manage instantiating a single Publisher object.

In your thread, as you are doing, publish messages there - a slight ammendment to your post_event helper:

def post_event(message, data):
    wx.CallAfter(lambda *a: pub.sendMessage(message, data=data))

In your main thread - subscribe to those messages. I'd say it's usually easiest to have one handler per message, rather than sending two different messages to the same handler as you were, so I went for

pub.subscribe(self.on_status_result, 'prepare.running')
pub.subscribe(self.on_status_finished, 'prepare.complete')

You can leave on_status_result as it is and define a similar on_status_finished. In my example, I had this later one enabling a new button to let you do some real work.

N.B. You need to be careful when naming the payload of your messages - pubsub infers quite a bit of information about what it's expecting there and it caught me at first.

---

P.S. Right at the end of preparing this answer - I found this blog post. It says something similar to what I have above, so I won't reproduce it, but they use the other method of instantiating Publisher() like your original example - implying that should work too. You may prefer the wording there. Simlarly - you might find this wxPython wiki page useful.

Answer 2

the problem is that the event system ends up calling the update function(event handler) from the threads themselves , you should pretty much never do that(basically you end up with strange race conditions and artifacts) ... always make the callback in the main thread.

wxPython has taken this into consideration and any methods called with wx.CallAfter will be called from the main program loop which is always running in the main thread. this combined with the wx.pubsub module allow you to create your own event frame work easily ... something like this

def MyPostEvent(event_name,event_data):
  #just a helper that triggers the event with wx.CallAfter
  wx.CallAfter(lambda *a:Publisher().sendMessage(event_name,data=event_data))

#then to post an event

MyPostEvent("some_event.i_made_up",{"payload":True})

#then in your main thread subscribe 

def OnEventHandler(evt):
  print "EVT.data",evt.data

pub = Publisher()
pub.subscribe("some_event.i_made_up",OnEventHandler)