Tkinter objects being garbage collected from the wrong thread

Question

I seem to be breaking tkinter on linux by using some multi-threading. As far as I can see, I am managing to trigger a garbage collection on a thread which is not the main GUI thread. This is causing __del__ to be run on a tk.StringVar instance, which tries to call the tcl stack from the wrong thread, causing chaos on linux.

The code below is the minimal example I've been able to come up with. Notice that I'm not doing any real work with matplotlib, but I can't trigger the problem otherwise. The __del__ method on Widget verifies that the Widget instance is being deleted from the other thread. Typical output is:

Running off thread on 140653207140096
Being deleted... <__main__.Widget object .!widget2> 140653210118576
Thread is 140653207140096
... (omitted stack from from `matplotlib`
  File "/nfs/see-fs-02_users/matmdpd/anaconda3/lib/python3.6/site-packages/matplotlib/text.py", line 218, in __init__
    elif is_string_like(fontproperties):
  File "/nfs/see-fs-02_users/matmdpd/anaconda3/lib/python3.6/site-packages/matplotlib/cbook.py", line 693, in is_string_like
    obj + ''
  File "tk_threading.py", line 27, in __del__
    traceback.print_stack()
...
Exception ignored in: <bound method Variable.__del__ of <tkinter.StringVar object at 0x7fec60a02ac8>>
Traceback (most recent call last):
  File "/nfs/see-fs-02_users/matmdpd/anaconda3/lib/python3.6/tkinter/__init__.py", line 335, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
_tkinter.TclError: out of stack space (infinite loop?)

By modifying the tkinter library code, I can verify that __del__ is being called from the same place as Widget.__del__.

Is my conclusion here correct? How can I stop this happening??

I really, really want to call matplotlib code from a separate thread, because I need to produce some complex plots which are slow to render, so making them off-thread, generating an image, and then displaying the image in a tk.Canvas widget seemed like an elegant solution.

Minimal example:

import tkinter as tk
import traceback
import threading

import matplotlib
matplotlib.use('Agg')
import matplotlib.figure as figure
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas

class Widget(tk.Frame):
    def __init__(self, parent):
        super().__init__(parent)
        self.var = tk.StringVar()
        #tk.Entry(self, textvariable=self.var).grid()
        self._thing = tk.Frame(self)
        def task():
            print("Running off thread on", threading.get_ident())
            fig = figure.Figure(figsize=(5,5))
            FigureCanvas(fig)
            fig.add_subplot(1,1,1)
            print("All done off thread...")
        #import gc
        #gc.collect()
        threading.Thread(target=task).start()

    def __del__(self):
        print("Being deleted...", self.__repr__(), id(self))
        print("Thread is", threading.get_ident())
        traceback.print_stack()

root = tk.Tk()
frame = Widget(root)
frame.grid(row=1, column=0)

def click():
    global frame
    frame.destroy()
    frame = Widget(root)
    frame.grid(row=1, column=0)

tk.Button(root, text="Click me", command=click).grid(row=0, column=0)

root.mainloop()

Notice that in the example, I don't need the tk.Entry widget. However if I comment out the line self._thing = tk.Frame(self) then I cannot recreate the problem! I don't understand this...

If I uncomment then gc lines, then also the problem goes away (which fits with my conclusion...)

Update: This seem to work the same way on Windows. tkinter on Windows seems more tolerant of being called on the "wrong" thread, so I don't get the _tkinter.TclError exception. But I can see the __del__ destructor being called on the non-main thread.

Answer 1

I had exactly the same problem

It was a nightmare to find the cause of the issue. I exaustivelly verified that no tkinter object was being called from any thread. I made a mechanism based in queues to handle tkinter objects in threads. There are many examples on the web on how to do that, or... search for a module 'mttkinter', a thread safe wrapper for Tkinter)

In a effort to force garbage collection, I used the "gc" method in the exit function of every TopLevel window of my App.

#garbage collector
import gc

...

gc.collect()

but for some reason, closing a toplevel window continued to reproduce the problem. Anyway... it was precisely using some prints in the aforementioned "mttkinter" module that I detected that, in spite the widgets are being created in the main thread, they could be garbage collected when garbage collector is triggered inside another thread. It looks like that the garbage collector gathers all the garbage without any distinction of its provenience (mainthread or other threads?). Someone please correct me if I'm wrong.

My solution was to call the garbage collector explicitly using the queue as well.

PutInQueue(gc.collect)

Where "PutInQueue" belongs to a module created by me to handle tkinter object and other kind of objects with thread safety.

Hope this report can be of great usefullness to someone or, it it is the case, to expose any eventual bug in garbage collector.