Matplotlib interactive event loops

Question

How does Matplotlib set up the event loop for backend libraries such as Qt while still allowing interaction via the python REPL? At least for Qt the main event loop must run in the main thread, but that's where the REPL is, right, so I'm struggling to see how the two can coexist.

My current attempt starts a QApplication in a separate Python threading.Thread

def mainloop():
    app = QtWidgets.QApplication([])
    while True:
        app.processEvents()
        time.sleep(0.01)
t = threading.Thread(target=mainloop)
t.daemon = True
t.start()

which sort-of works, but I get this warning and it sometimes crashes:

QObject: Cannot create children for a parent that is in a different thread.
(Parent is QApplication(0x7fc5cc001820), parent's thread is QThread(0x7fc5cc001a40), current thread is QThread(0x2702160)

Update 1

Here's an attempt using QThread:

from PyQt5 import QtGui, QtCore, QtWidgets
import time

class Mainloop(QtCore.QObject):
    def __init__(self):
        super().__init__()
        self.app = QtWidgets.QApplication([])

    def run(self):
        while True:
            self.app.processEvents()
            time.sleep(1)

t = QtCore.QThread()
m = Mainloop()
m.moveToThread(t)
t.started.connect(m.run)
t.start()

# Essentially I want to be able to interactively build a GUI here
dialog = QtWidgets.QDialog()
dialog.show()

Update 2

Basically, I want to emulate the following interactive python session, that is, not running it as a script to present a ready made GUI. What is the magic that keeps the appearance of the figure window from blocking the python interpreter?

Python 2.7.13 (default, Jan 19 2017, 14:48:08) 
Type "copyright", "credits" or "license" for more information.

IPython 5.2.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import matplotlib.pyplot as plt

In [2]: plt.ion()

In [3]: fig = plt.figure()

In [4]: # The last command opens a figure window which remains responsive.
   ...:  I can resize it and the window redraws, yet I can still interact 
   ...: with the python interpreter and interactively add new content to t
   ...: he figure

In [5]: ax = fig.add_subplot(111)

In [6]: # The last command updated the figure window and I can still inter
   ...: act with the interpreter

In [7]: 

Answer 1

The magic is not done by Matplotlib but by IPython, in an actually called magic command.

If you start IPython with ipython --gui=qt, or type %gui qt, the Qt event loop will be started and integrated into IPython (the --matplotlib option does this as well but for the default backend). After that you can just create widgets on the command line without having to start the event loop.

~> ipython
Python 3.5.3 |Continuum Analytics, Inc.| (default, Mar  6 2017, 12:15:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %gui qt

In [2]: from PyQt5 import QtWidgets

In [3]: win = QtWidgets.QPushButton("click me")

In [4]: win.show()

In [5]: win.raise_()

There is a short section on Integrating with GUI event loop that explains how this works under the hood. But just to be clear, you don't have to follow those instructions because event loop integration has already been implemented for Qt. Just use the %gui qt magic-command.

Update

So indeed you can do this without IPython as well. PyQt makes it possible to have both a regular Python shell and a running event loop simultaneously, as explained in the section on Using PyQt5 from the Python Shell from the PyQt reference guide.

Small difference is that you need to explicitly create the QApplication yourself. For instance type:

~> python
Python 3.6.0 |Continuum Analytics, Inc.| (default, Dec 23 2016, 13:19:00)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.QtWidgets import QApplication, QWidget
>>> a = QApplication([])
>>> w = QWidget()
>>> w.show()
>>> w.raise_()

I was wondering how this works with Matplotlib and looked around a bit in the source code of pyplot. It seems to me that Matplotlib can handle both situations. When you execute plt.ion() a install_repl_displayhook function is called. Its doc-string says:

Install a repl display hook so that any stale figure are automatically redrawn when control is returned to the repl. This works with IPython terminals and kernels, as well as vanilla python shells.

So, even though IPython is not a dependency of Matplotlib, Matplotlib is aware of IPython and can detect if it is in the IPython shell or in a regular Python shell.