Understanding performance limitations of the Tkinter Canvas

Question

I've created a simple application to display a scatterplot of data using Tkinter's Canvas widget (see the simple example below). After plotting 10,000 data points, the application becomes very laggy, which can be seen by trying to change the size of the window.

I realize that each item added to the Canvas is an object, so there may be some performance issues at some point, however, I expected that level to be much higher than 10,000 simple oval objects. Further, I could accept some delays when drawing the points or interacting with them, but after they are drawn, why would just resizing the window be so slow?

After reading effbot's performance issues with the Canvas widget it seems there may be some unneeded continuous idle tasks during resizing that need to be ignored:

The Canvas widget implements a straight-forward damage/repair display model. Changes to the canvas, and external events such as Expose, are all treated as “damage” to the screen. The widget maintains a dirty rectangle to keep track of the damaged area.

When the first damage event arrives, the canvas registers an idle task (using after_idle) which is used to “repair” the canvas when the program gets back to the Tkinter main loop. You can force updates by calling the update_idletasks method.

So, the question is whether there is any way to use update_idletasks to make the application more responsive once the data has been plotted? If so, how?

Below is the simplest working example. Try resizing the window after it loads to see how laggy the application becomes.

Update

I originally observed this problem in Mac OS X (Mavericks), where I get a substantial spike in CPU usage when just resizing the window. Prompted by Ramchandra's comments I've tested this in Ubuntu and this doesn't seem to occur. Perhaps this is a Mac Python/Tk problem? Wouldn't be the first I've run into, see my other question:

PNG display in PIL broken on OS X Mavericks?

Could someone also try in Windows (I don't have access to a Windows box)?

I may try running on the Mac with my own compiled version of Python and see if the problem persists.

Minimal working example:

import Tkinter
import random

LABEL_FONT = ('Arial', 16)


class Application(Tkinter.Frame):
    def __init__(self, master, width, height):
        Tkinter.Frame.__init__(self, master)
        self.master.minsize(width=width, height=height)
        self.master.config()
        self.pack(
            anchor=Tkinter.NW,
            fill=Tkinter.NONE,
            expand=Tkinter.FALSE
        )

        self.main_frame = Tkinter.Frame(self.master)
        self.main_frame.pack(
            anchor=Tkinter.NW,
            fill=Tkinter.NONE,
            expand=Tkinter.FALSE
        )

        self.plot = Tkinter.Canvas(
            self.main_frame,
            relief=Tkinter.RAISED,
            width=512,
            height=512,
            borderwidth=1
        )
        self.plot.pack(
            anchor=Tkinter.NW,
            fill=Tkinter.NONE,
            expand=Tkinter.FALSE
        )
        self.radius = 2
        self._draw_plot()

    def _draw_plot(self):

        # Axes lines
        self.plot.create_line(75, 425, 425, 425, width=2)
        self.plot.create_line(75, 425, 75, 75, width=2)

        # Axes labels
        for i in range(11):
            x = 75 + i*35
            y = x
            self.plot.create_line(x, 425, x, 430, width=2)
            self.plot.create_line(75, y, 70, y, width=2)
            self.plot.create_text(
                x, 430,
                text='{}'.format((10*i)),
                anchor=Tkinter.N,
                font=LABEL_FONT
            )
            self.plot.create_text(
                65, y,
                text='{}'.format((10*(10-i))),
                anchor=Tkinter.E,
                font=LABEL_FONT
            )

        # Plot lots of points
        for i in range(0, 10000):
            x = round(random.random()*100.0, 1)
            y = round(random.random()*100.0, 1)

            # use floats to prevent flooring
            px = 75 + (x * (350.0/100.0))
            py = 425 - (y * (350.0/100.0))

            self.plot.create_oval(
                px - self.radius,
                py - self.radius,
                px + self.radius,
                py + self.radius,
                width=1,
                outline='DarkSlateBlue',
                fill='SteelBlue'
            )

root = Tkinter.Tk()
root.title('Simple Plot')

w = 512 + 12
h = 512 + 12

app = Application(root, width=w, height=h)
app.mainloop()

Answer 1

There is actually a problem with some distributions of TKinter and OS Mavericks. Apparently you need to install ActiveTcl 8.5.15.1. There is a bug with TKinter and OS Mavericks. If it still isn't fast eneough, there are some more tricks below.

You could still save the multiple dots into one image. If you don't change it very often, it should still be faster. If you are changing them more often, here are some other ways to speed up a python program. This other stack overflow thread talks about using cython to make a faster class. Because most of the slowing down is probably due to the graphics this probably won't make it a lot faster but it could help.

Suggestions on how to speed up a distance calculation

you could also speed up the for loop by defining an iterator ( ex: iterator = (s.upper() for s in list_to_iterate_through) ) beforehand, but this is called to draw the window, not constantly as the window is maintained, so this shouldn't matter very much. Also, a another way to speed things up, taken from python docs, is to lower the frequency of python's background checks:

"The Python interpreter performs some periodic checks. In particular, it decides whether or not to let another thread run and whether or not to run a pending call (typically a call established by a signal handler). Most of the time there's nothing to do, so performing these checks each pass around the interpreter loop can slow things down. There is a function in the sys module, setcheckinterval, which you can call to tell the interpreter how often to perform these periodic checks. Prior to the release of Python 2.3 it defaulted to 10. In 2.3 this was raised to 100. If you aren't running with threads and you don't expect to be catching many signals, setting this to a larger value can improve the interpreter's performance, sometimes substantially."

Another thing I found online is that for some reason setting the time by changing os.environ['TZ'] will speed up the program a small amount.

If this still doesn't work, than it is likely that TKinter is not the best program to do this in. Pygame could be faster, or a program that uses the graphics card like open GL (I don't think that is available for python, however)

Answer 2

Tk must be getting bogged down looping over all of those ovals. I'm not sure that the canvas was ever intended to hold so many items at once.

One solution is to draw your plot into an image object, then place the image into your canvas.