Matplotlib with multiprocessing freeze computer

Question

I have an issue with matplotlib and multiprocessing. I launch a first process, where I display an image and select an area, and close the figure. Then I launch another process, where I call a graph function that is regularly updated. Up this point, eveything works fine. Then when I try to launch another process with the SAME graph function, it freeze my whole computer, BUT the background processes stil work... I only have one of these errors (it's not always the same):

error 1 :

XIO: fatal IO error 25 (Inappropriate ioctl for device) on X server ":0.0" after 4438 requests (4438 known processed) with 30 events remaining. XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0.0" after 4443 requests (4443 known processed) with 31 events remaining. [xcb] Unknown sequence number while processing queue [xcb] Most likely this is a multi-threaded client and XInitThreads has not been called [xcb] Aborting, sorry about that. python: ../../src/xcb_io.c:274: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.

error 2 :

X Error of failed request: BadIDChoice (invalid resource ID chosen for this connection) Major opcode of failed request: 53 (X_CreatePixmap) Resource id in failed request: 0x5600299 Serial number of failed request: 4793 Current serial number in output stream: 4795 XIO: fatal IO error 25 (Inappropriate ioctl for device) on X server ":0.0" after 4788 requests (4788 known processed) with 31 events remaining. XIO: fatal IO error 25 (Inappropriate ioctl for device) on X server ":0.0" after 4793 requests (4793 known processed) with 32 events remaining.

The weird part is that I can totaly launch several process calling the graph function without any issue, it's the coupling with the first plot that make it unstable.

When trying to debug, I found out that a simple fig=plt.figure() is enough to crash everything : in fact, any call to plt ...

I read here and there that you can force matplotlib to use the agg backend and it helps with the multiprocess, but some widgets doesn't work with it so I would like to avoid this.

I don't really understand why using matplotlib in differents processes could cause problems, so if anyone could explain the reasons and/or help me with a workaround, it would be very nice.

Answer 1

I just had a very similar issue in which I have a class which produces plots in parallel. The first time I create a new instance of that class and run the plotting function, everything works perfectly. But if I create a new instance and plot, everything freezes.

I fixed it by writing a bash script which will in turn run a python script with the code for a single class instantiation + plot call. In other words, closing python between one plot call and the next one makes a clean slate of your working environment the computer does not freeze anymore. This is not an optimal solution, but it's working :)