How to speed up matplotlib when plotting and saving lots of figures?

Question

I am processing observing data from many antenna baseline. Currently what I am working is to plot ~ 40 figures, each of which has 4x5 subplot region. I found it slow when plotting and saving figures with matplotlib in loops. Here are my codes:

    import numpy as np
    import matplotlib.pyplot as plt
    import time
    ...

    PLT_PAGE_NUM = 39 # default is 39
    SUB_PLT_NUM = 20 # default is 20

    for pp in xrange(0,PLT_PAGE_NUM):

        plt.figure(figsize=(20,12))

        start_time = time.clock() 
        for kk in xrange(0,SUB_PLT_NUM):
            plt.subplot(5,4,kk+1)
            plt.plot(np.arange(0,TIME_LENGTH), xcor_real_arr[20*pp+kk,0:],'r-',
                     range(0,TIME_LENGTH), xcor_imag_arr[20*pp+kk,0:],'b-')
            plt.title('XCOR of '+ ind_arr[20*pp+kk], color='k') 

        plt.savefig('test_imag_real'+str(pp)+'.png',format='png',dpi=100)
        print 'Fig-'+str(pp)+' has been saved'
        print "Excution time:", time.clock()-start_time

And the excution time info is:

######### Check your inputs setting #########
You have selected 2 files.
The time interval is From 2011-10-20_14:28:38 to 2011-10-20_15:10:54
Your time resolution is set to 1.125s
The total plot points number is: 100
Your frequency channel is: ch2
######### Hardworking...please wait #########
Fig-0 has been saved
Excution time: *2.52576639619*
Fig-1 has been saved
Excution time: *2.59867230708*
Fig-2 has been saved
Excution time: *2.81915188482*
Fig-3 has been saved
Excution time: *2.83102198991*
Program ends

As you have seen, I just plot 4 figures which cost about 11 seconds. It takes ~2 minutes to plot and save all 39 figures. I dont' know where the bottleneck is. May you help to make it faster? Thanks!

Answer 1

I've modified your code to make it runnable:

import numpy as np
import matplotlib.pyplot as plt
import time

PLT_PAGE_NUM = 39 # default is 39
SUB_PLT_NUM = 20 # default is 20
TIME_LENGTH = 1000

xcor_real_arr = np.random.random((SUB_PLT_NUM*PLT_PAGE_NUM,TIME_LENGTH))
xcor_imag_arr = np.random.random((SUB_PLT_NUM*PLT_PAGE_NUM,TIME_LENGTH))
for pp in xrange(0,PLT_PAGE_NUM):

    plt.figure(figsize=(20,12))

    start_time = time.time() 
    for kk in xrange(0,SUB_PLT_NUM):
        plt.subplot(5,4,kk+1)
        plt.plot(np.arange(0,TIME_LENGTH), xcor_real_arr[SUB_PLT_NUM*pp+kk,0:],'r-',
                 range(0,TIME_LENGTH), xcor_imag_arr[SUB_PLT_NUM*pp+kk,0:],'b-')
        plt.title('XCOR of '+ str(SUB_PLT_NUM*pp+kk), color='k') 

    plt.savefig('test_imag_real'+str(pp)+'.png',format='png',dpi=100)
    print 'Fig-'+str(pp)+' has been saved'
    print "Excution time:", time.time()-start_time

On my machine, each figure takes about 3 seconds:

Fig-0 has been saved
Excution time: 3.01798415184
Fig-1 has been saved
Excution time: 3.08960294724
Fig-2 has been saved
Excution time: 2.9629740715

Using ideas from the Matplotlib Animations Cookbook (and also demonstrated by Joe Kington, here), we can speed this up by about 33% (1 second per figure) by reusing the same axes and simply redefining the y-data for each plot:

import numpy as np
import matplotlib.pyplot as plt
import time

PLT_PAGE_NUM = 39 # default is 39
SUB_PLT_NUM = 20 # default is 20
TIME_LENGTH = 1000

xcor_real_arr = np.random.random((SUB_PLT_NUM*PLT_PAGE_NUM,TIME_LENGTH))
xcor_imag_arr = np.random.random((SUB_PLT_NUM*PLT_PAGE_NUM,TIME_LENGTH))
plt.figure(figsize=(20,12))

ax = {}
line1 = {}
line2 = {}

for pp in xrange(0,PLT_PAGE_NUM):
    start_time = time.time() 
    for kk in xrange(0,SUB_PLT_NUM):
        if pp == 0:
            ax[kk] = plt.subplot(5,4,kk+1)
            line1[kk], line2[kk] = ax[kk].plot(np.arange(0,TIME_LENGTH),
                                   xcor_real_arr[SUB_PLT_NUM*pp+kk,0:],'r-',
                                   range(0,TIME_LENGTH),
                                   xcor_imag_arr[SUB_PLT_NUM*pp+kk,0:],'b-')
        else:
            line1[kk].set_ydata(xcor_real_arr[SUB_PLT_NUM*pp+kk,0:])
            line2[kk].set_ydata(xcor_imag_arr[SUB_PLT_NUM*pp+kk,0:])
        plt.title('XCOR of '+ str(SUB_PLT_NUM*pp+kk), color='k') 

    plt.savefig('test_imag_real'+str(pp)+'.png',format='png',dpi=100)
    print 'Fig-'+str(pp)+' has been saved'
    print "Excution time:", time.time()-start_time

which yields these execution times:

Fig-0 has been saved
Excution time: 3.0408449173
Fig-1 has been saved
Excution time: 2.05084013939
Fig-2 has been saved
Excution time: 2.01951694489

(The first figure still takes 3 seconds to set up the initial plots. It is on subsequent figures where we can save some time.)