PyQt: Occasional segfaults when using QApplication.quit

Question

Note: I also posted this on the PyQt mailinglist -- I'll answer my own question here if a good answer comes up there.

I'm having problems with occasional segfaults when executing QApplication.quit(), possibly related to libQt5Network.so and/or QtWebkit.

First of all, the 3 test systems I'm using:

• Arch Linux, PyQt 5.2, Qt 5.2.0, Python 3.3.3
• Ubuntu 13.10, PyQt 5.0.1, Qt 5.0.2, Python 3.3.2 in a VM
• Windows 7, PyQt 5.2, Qt 5.2.0, Python 3.3.3

These crashes never happened on Arch for me so far, quite often on Ubuntu, and from time to time in Windows. (though Windows is just a guess, I just get this python.exe is not working anymore foo.)

Original crash

I first noticed the problem in a big(ger) project, qutebrowser, where it gave me this stacktrace when typing :quit (on Ubuntu):

#0  0xb5c296fc in QMutex::lock() () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#1  0xb3bdd97d in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#2  0xb3bdf0d0 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#3  0xb3bd4418 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#4  0xb3bd8b1e in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#5  0xb5dedf10 in QMetaObject::activate(QObject*, int, int, void**) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#6  0xb5dee48b in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#7  0xb5e59155 in QIODevice::readyRead() ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#8  0xb3bb1f14 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#9  0xb3ba4d99 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#10 0xb3bc03bb in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#11 0xb6483a54 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
   from /usr/lib/i386-linux-gnu/libQt5Widgets.so.5
#12 0xb6488e66 in QApplication::notify(QObject*, QEvent*) ()
   from /usr/lib/i386-linux-gnu/libQt5Widgets.so.5
#13 0xb6bb7e80 in sipQApplication::notify(QObject*, QEvent*) ()
   from /usr/lib/python3/dist-packages/PyQt5/QtWidgets.cpython-33m-i386-linux-gnu.so
#14 0xb5dc737a in QCoreApplication::notifyInternal(QObject*, QEvent*) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#15 0xb5e11f67 in ?? () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#16 0xb5aaf83e in g_main_context_dispatch ()
   from /lib/i386-linux-gnu/libglib-2.0.so.0
#17 0xb5aafbe8 in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
#18 0xb5aafca8 in g_main_context_iteration ()
   from /lib/i386-linux-gnu/libglib-2.0.so.0
#19 0xb5e1138f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#20 0xb5dc5c06 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#21 0xb5dc6014 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#22 0xb5c2b90b in QThread::exec() ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#23 0xb5c2b99b in QThread::run() () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#24 0xb5c2fa08 in ?? () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#25 0xb7774d78 in start_thread (arg=0xa5314b40) at pthread_create.c:311
#26 0xb76ac01e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:131

Core dump here (15MB, gzip).

Minimal example

Then I tried again with a minimal example which quits itself automatically after a second with a QTimer. I had to run it in a loop for about a minute or so before it happened:

from PyQt5.QtCore import QUrl, QTimer
from PyQt5.QtWidgets import QApplication
from PyQt5.QtWebKitWidgets import QWebView

app = QApplication([])
wv = QWebView()
wv.load(QUrl("http://www.heise.de/"))
t = QTimer()
t.timeout.connect(QApplication.quit)
t.start(1000)
wv.show()
app.exec_()

This gave me this very similiar stacktrace (on Ubuntu):

#0  0xb6cfd8d2 in QCoreApplication::postEvent(QObject*, QEvent*, int) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#1  0xb6d21c83 in QMetaObject::activate(QObject*, int, int, void**) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#2  0xb6d2248b in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#3  0xb3e47935 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#4  0xb3dcf687 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#5  0xb3e483b3 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#6  0xb6d21f10 in QMetaObject::activate(QObject*, int, int, void**) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#7  0xb6d2248b in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#8  0xb3e43fe5 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#9  0xb3d93b1e in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#10 0xb3d94630 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#11 0xb3d9471b in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#12 0xb6d21f10 in QMetaObject::activate(QObject*, int, int, void**) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#13 0xb6d2248b in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#14 0xb6d8d155 in QIODevice::readyRead() ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#15 0xb3e09f14 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#16 0xb3dfcd99 in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#17 0xb3e183bb in ?? () from /usr/lib/i386-linux-gnu/libQt5Network.so.5
#18 0xb492ba54 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
   from /usr/lib/i386-linux-gnu/libQt5Widgets.so.5
#19 0xb4930e66 in QApplication::notify(QObject*, QEvent*) ()
   from /usr/lib/i386-linux-gnu/libQt5Widgets.so.5
#20 0xb505fe80 in sipQApplication::notify(QObject*, QEvent*) ()
   from /usr/lib/python3/dist-packages/PyQt5/QtWidgets.cpython-33m-i386-linux-gnu.so
#21 0xb6cfb37a in QCoreApplication::notifyInternal(QObject*, QEvent*) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#22 0xb6d45f67 in ?? () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#23 0xb65f483e in g_main_context_dispatch ()
   from /lib/i386-linux-gnu/libglib-2.0.so.0
#24 0xb65f4be8 in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
#25 0xb65f4ca8 in g_main_context_iteration ()
   from /lib/i386-linux-gnu/libglib-2.0.so.0
#26 0xb6d4536d in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#27 0xb6cf9c06 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#28 0xb6cfa014 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#29 0xb6b5f90b in QThread::exec() ()
   from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#30 0xb6b5f99b in QThread::run() () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#31 0xb6b63a08 in ?? () from /usr/lib/i386-linux-gnu/libQt5Core.so.5
#32 0xb7798d78 in start_thread (arg=0xa7812b40) at pthread_create.c:311
#33 0xb76d001e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:131

Coredump here (15MB, gzip).

Anyone has an idea what's going wrong there? Some magic with stuff being garbage-collected the wrong way around? I also tried some workarounds* for similiar sympthoms on PyQt4, but that didn't help either.

* can't find the StackOverflow-answer where it was described -- basically setting QtWidgets.qApp to the QApplication instance before running exec_(), and to None afterwards.

Answer 1

I've been working on catching segfaults in PyQt / PySide for quite some time. Basically I found that most of the segfaults can be blamed on the async nature of the library(thus on us as well).

For your exact example, you've connected the timeout signal to the quit method. What could happen here is that in the event of a timeout, quit gets called, and when the process is terminated, suddenly all references to the application's objects are invalid. But while this operation was happening, QT's event loop was still running, and it tried to access it's QNetworkAccessManager to dispatch another signal, but the reference to that memory location was already invalid, thus the segfault happened.

What needs to be done in these cases, is that you implement a kind of a shutdown method, that will make sure all operations are stopped, delete the QT components you have in use in the right order, and then, just then, call allow exiting.

I wrote about the subject in more detail here, including a safe shutdown method for QtWebKit apps: https://github.com/integricho/path-of-a-pyqter/tree/master/qttut08