Why can't matplotlib plot in a different thread?

Question

Minimum working example

I expect the following to show a plot, but i see no plot and the interpreter just hangs (my backend reports itself as TkAgg).

import matplotlib.pyplot as plt from threading import Thread  def plot():     fig, ax = plt.subplots()     ax.plot([1,2,3], [1,2,3])     plt.show()  def main():     thread = Thread(target=plot)     thread.setDaemon(True)     thread.start()     print 'Done' 

How do I get the plot to display?

Context

I am running a simulation with lots iterations and would like to update my plot every 1000 iterations so that I can monitor how my simulation is evolving.

Psuedocode below:

iterations = 100000 for i in iterations:     result = simulate(iteration=i)     if not i % 1000:         # Update/redraw plot here:         # Add some lines, add some points, reset axis limits, change some colours 

Having the plot in the main thread causes the plot GUI to hang/crash presumably because I have other work going on. So the idea was to do the plotting in a separate thread.

I have seen suggestions (e.g. here) to use a process rather than a thread. But then I cannot manipulate the figure or axes to add lines etc while my simulation runs because the figure object is in the remote process.

Edit

I'm not convinced this question is a duplicate of another one because that question deals with why the pyplot api cannot be used to manipulate two different plots that are each on a separate thread. It is because race conditions arising from executing two plots simultaneously prevents pyplot from determining which figure is the current figure.

However, I only have 1 plot and so pyplot only ever has a single and unique current figure.

Answer 1

As other people have told, Matplotlib is not thread safe, one option you have is to use multiprocessing. You say that this is not good for you, because you need access to the axes from different process, but you can overcome this by sharing data between the simulation process and the root process and then managing all the plotting related activities in the root process. For example

import matplotlib matplotlib.use('TkAgg') from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2TkAgg import multiprocessing import time import random from Tkinter import *   #Create a window window=Tk()    def main():     #Create a queue to share data between process     q = multiprocessing.Queue()      #Create and start the simulation process     simulate=multiprocessing.Process(None,simulation,args=(q,))     simulate.start()      #Create the base plot     plot()      #Call a function to update the plot when there is new data     updateplot(q)      window.mainloop()     print 'Done'   def plot():    #Function to create the base plot, make sure to make global the lines, axes, canvas and any part that you would want to update later      global line,ax,canvas     fig = matplotlib.figure.Figure()     ax = fig.add_subplot(1,1,1)     canvas = FigureCanvasTkAgg(fig, master=window)     canvas.show()     canvas.get_tk_widget().pack(side=TOP, fill=BOTH, expand=1)     canvas._tkcanvas.pack(side=TOP, fill=BOTH, expand=1)     line, = ax.plot([1,2,3], [1,2,10])     def updateplot(q):     try:       #Try to check if there is data in the queue         result=q.get_nowait()          if result !='Q':              print result                  #here get crazy with the plotting, you have access to all the global variables that you defined in the plot function, and have the data that the simulation sent.              line.set_ydata([1,result,10])              ax.draw_artist(line)              canvas.draw()              window.after(500,updateplot,q)         else:              print 'done'     except:         print "empty"         window.after(500,updateplot,q)   def simulation(q):     iterations = xrange(100)     for i in iterations:         if not i % 10:             time.sleep(1)                 #here send any data you want to send to the other process, can be any pickable object             q.put(random.randint(1,10))     q.put('Q')  if __name__ == '__main__':     main() 

Answer 2

I had a similar problem where I wanted to update a mapltolib plot from a different thread, and I am posting my solution here in case others have a similar problem in the future.

As noted the tkagg are not threading safe so you must make sure all calls to matplotlib are from a single thread. This means that the threads must communicate, so that the 'plotting thread' always executes matplotlib functions.

My solution was to create a decorator, that will execute all decorated functions in the 'plotting thread', and then to decorate all the relevant functions. This allows you to do what you want without any change to syntax in the main code.

i.e. when you call ax.plot(...) in one thread, you will have it automatically executed in a different thread.

import matplotlib.pyplot as plt import matplotlib import threading import time import queue import functools   #ript(Run In Plotting Thread) decorator def ript(function):     def ript_this(*args, **kwargs):         global send_queue, return_queue, plot_thread         if threading.currentThread() == plot_thread: #if called from the plotting thread -> execute             return function(*args, **kwargs)         else: #if called from a diffrent thread -> send function to queue             send_queue.put(functools.partial(function, *args, **kwargs))             return_parameters = return_queue.get(True) # blocking (wait for return value)             return return_parameters     return ript_this  #list functions in matplotlib you will use functions_to_decorate = [[matplotlib.axes.Axes,'plot'],                          [matplotlib.figure.Figure,'savefig'],                          [matplotlib.backends.backend_tkagg.FigureCanvasTkAgg,'draw'],                          ] #add the decorator to the functions for function in functions_to_decorate:     setattr(function[0], function[1], ript(getattr(function[0], function[1])))  # function that checks the send_queue and executes any functions found def update_figure(window, send_queue, return_queue):     try:         callback = send_queue.get(False)  # get function from queue, false=doesn't block         return_parameters = callback() # run function from queue         return_queue.put(return_parameters)     except:         None     window.after(10, update_figure, window, send_queue, return_queue)  # function to start plot thread def plot():     # we use these global variables because we need to access them from within the decorator     global plot_thread, send_queue, return_queue     return_queue = queue.Queue()     send_queue = queue.Queue()     plot_thread=threading.currentThread()     # we use these global variables because we need to access them from the main thread     global ax, fig     fig, ax = plt.subplots()     # we need the matplotlib window in order to access the main loop     window=plt.get_current_fig_manager().window     # we use window.after to check the queue periodically     window.after(10, update_figure, window, send_queue, return_queue)     # we start the main loop with plt.plot()     plt.show()   def main():     #start the plot and open the window     thread = threading.Thread(target=plot)     thread.setDaemon(True)     thread.start()     time.sleep(1) #we need the other thread to set 'fig' and 'ax' before we continue     #run the simulation and add things to the plot     global ax, fig     for i in range(10):         ax.plot([1,i+1], [1,(i+1)**0.5])         fig.canvas.draw()         fig.savefig('updated_figure.png')         time.sleep(1)     print('Done')     thread.join() #wait for user to close window main() 

Note that if you forget to decorate any functions, you may get a segmentation fault.

Also, in this example the child thread handles the plot and the main thread the simulation. In general it is advised to do the reverse, (i.e. let the main thread have the graphics).