Why does this loop in python runs progressively slower?

Question

In this code, there is a 4-D array of 13x13 images. I would like to save each 13x13 image using matplotlib.pyplot. Here for debugging purposes, I limit the outer loop to 1.

#fts is a numpy array of shape (4000,100,13,13)
no_images = 4000
for m in [1]:  
    for i in range(no_images):
        print i,
        fm = fts[i][m]
        if fm.min() != fm.max():
            fm -= fm.min()
            fm /= fm.max()  #scale to [0,1]
        else:
            print 'unscaled'
        plt.imshow(fmap)
        plt.savefig('m'+str(m)+'_i'+str(i)+'.png')

Saving 4000 images took more than 20 hours. Why is it this slow? If I limit the inner loop to the first 100 images, it takes about 1 minute. So the whole thing should be completed in 40 minutes, not over 20 hours! And I notice it seems to run progressively slower.

Answer 1

What you experience here is a memory leak: you keep creating instances of AxesImage objects (by repetitively calling plt.imshow) to the moment they can't fit into RAM; and then the whole thing begins swapping to disk, which is incredibly slow. To avoid memory leaks, you can either destroy AxesImage instance when you don't need it:

...
image = plt.imshow(fmap)
plt.savefig('m'+str(m)+'_i'+str(i)+'.png')
del(image)

Or, alternatively, you can create only one AxesImage, and then just change the data in it:

...
image = None
for m in [1]:  
    for i in range(no_images):
        ...
        if image is None:
             image = plt.imshow(fmap)
        else:
             image.set_data(fmap)
        ...