Clean separation between application thread and Qt thread (Python - PyQt)

Question

I prefer to write my application without even thinking about a graphical user interface. Once the application code is working properly, I like to glue a GUI layer on top of it - with a clean interface between the two.

I first tried to make the GUI run in a different process from the application. But I soon regretted that experiment. It is far from trivial to setup a communication link between two processes. So I decided that for now, multiple threads are fine (although the Python Global Interpreter Lock makes them run on a single core).

The MainThread is completely in the hands of the Qt GUI. Apparently this is standard practice. So let us suppose that the overall structure of the software should look like this (note that qtThread is synonymous to MainThread):

My application code is running in the appThread - cleanly separated from the GUI. But at some point, there has to be interaction.

I have read many articles about how to organize this, but many sources contradict each other. Even the official Qt application is wrong according to many people (the Official documentation encourages to subclass QThread). The most enlightening articles I could find are these:

http://ilearnstuff.blogspot.be/2012/08/when-qthread-isnt-thread.html http://ilearnstuff.blogspot.be/2012/09/qthread-best-practices-when-qthread.html

Even after considering all that, I still remain in doubt about several things.

Question 1. What is the most proper way to start the appThread?

---

What is the most proper way to start the appThread? Correct me if I am wrong, but I believe that there are two choices:

Choice 1: Start a standard Python thread
Python provides the threading library that one can import to spawn new threads:

import threading

if __name__ == '__main__':
    # 1. Create the qt thread (is MainThread in fact)
    qtApp = QApplication(sys.argv)
    QApplication.setStyle(QStyleFactory.create('Fusion'))

    # 2. Create the appThread
    appThread = threading.Thread(name='appThread', target=appThreadFunc, args=(p1,p2,))
    appThread.start()

    # 3. Start the qt event loop
    qtApp.exec_()
    print('Exiting program')

This choice looks the cleanest to me. You can truly write your appThread code without even thinking about a GUI. After all, you're using the standard Python threading library. There is no Qt stuff in there.
But I cannot find clear documentation about setting up a communication link between the appThread and the MainThread. More about that issue in the second question..

Choice 2: Start a QThread thread
This choice looks not so clean, because you have to mess with Qt stuff to write your application code. Anyway, it looks like a viable option, because the communication link between both threads - appThread and MainThread - is probably better supported.
There are myriads of ways to start a QThread thread. The official Qt documentation encouraged to subclass QThread and reimplement the run() method. But I read that this practice is in fact very bad. Refer for more info to the two links I've posted at the beginning of my question.

Question 2. What is the best communication link between both threads?

---

What is the best communication link between both threads? Obviously the answer to this question depends very much on the choice made in Question 1. I can imagine that linking a standard Python thread to the GUI differs very much from linking a QThread.
I will leave it up to you to make suggestions, but a few mechanisms that pop up in my mind are:

• Queues: using standard Python queues or Qt Queues?
• Signal/Slot mechanism: ...
• Sockets: should work, but looks a bit cumbersome
• Pipes: ...
• Temporary files: cumbersome

Notes :

---

Please mention if your answer applies to Python 2.x or 3.x. Also keep in mind that confusion may arise quickly when speaking about threads, queues and the like. Please mention if you refer to a standard Python thread or a QThread, a standard Python queue or a QQueue, ...

Answer 1

What I suggest is to do what most others do. Wait until there is code that needs to be run in a separate thread, and then only put that piece of code in a thread. There is no need for your code to be in a separate thread to have good code separation. The way I would do it is the following:

Have your appThread code (the code that has no knowledge of a GUI) in a base class that only has knowledge of non-GUI libraries. This makes it easy to support a command-line version of your code later as well. Put code that you need to execute asynchronously inside regular Python threads for this base class. Make sure the code you want to execute asynchronously is just a single function call to make it easier for my next point.

Then, have a child class in a separate file that inherits from both the base class you just wrote and the QMainWindow class. Any code that you need to run asynchronously can be called via the QThread class. If you made the code you want to run asynchronously available in one function call as I mentioned above, it's easy to make this step work for your QThread child class.

Why do the above?

It makes it much easier to manage state and communication. Why make yourself go insane with race conditions and thread communication when you don't have to? There's also really no performance reason to have separate threads in a GUI for app code vs GUI code since most of the time the user is not actually inputting much as far as the CPU is concerned. Only the parts that are slow should be put in threads, both to save sanity and to make code management easier. Plus, with Python, you don't gain anything from separate threads thanks to the GIL.