Creating a live plot of CSV data with Matplotlib

Question

I am trying to use Matplotlib to visualize some measurements. The measurements typically last about 24hrs and will include ~30k lines of data in the csv. I've been struggling mostly with getting my plots to actually animate. I can execute the code and it will display a snapshot up to the current point in time but nothing else. When I try to autoscale it, nothing plots and it just defaults to a view of -.6 to +.6 on both axes. Am I supposed to call plt.draw() for this to work? This is what I have so far:

import numpy as np
import datetime as dt
import matplotlib
import matplotlib.pyplot as plt
from matplotlib import animation

FMT = '%Y-%m-%d %H:%M:%S.%f'
data = np.genfromtxt('sampleData.csv', delimiter=',', skip_header=5,
                 names=['x', 'y', 'z', 'w'], dtype=['object', 'int8', 'int8', 'float'])                  

mytime = [dt.datetime.strptime(i.decode('ascii'), FMT) for i in data['x']]
thickness = data['w']

fig = plt.figure()
axes = fig.add_subplot(111)
line, = axes.plot([], [], '.')
plt.show(block=False)

def init():
    line.set_data([],[]) 
    return line,

fig.canvas.draw()

def animate(i):
    xdata = mytime[:i]
    ydata = thickness[:i]
    line.set_data(xdata, ydata)
    plt.draw()
    axes.relim()
    axes.autoscale(True,'both',True) 
    axes.autoscale_view(True,True,True)
    return line,

anim = animation.FuncAnimation(fig, animate, init_func=init,  
                           interval=0, blit=True)

plt.show()

This is a sample line of data from the CSV:

2013-09-25 14:51:15.329091,1,0,439.80,,,,0,0,

Answer 1

You have two issues. One is because you're effectively drawing things twice, the second is purely human psychology (the plot appears to slow over time because you're adding one point to 10000 vs adding one point to 10 or 100).

---

Let's discuss the double-draw first:

The FuncAnimation will draw things for you, but by telling it to use blitting, it's only updating the inside of the axes and not the ticks, etc. Therefore, you need to manually call draw, but the animation will be calling draw_artist, as well.

You should be able to get at least a 2x speedup by removing blit=True and plt.draw()

Furthermore, by setting interval=0, you're forcing it to draw constantly, which will effectively force things to lock up. Set the interval to something more reasonable, e.g. 25 (the interval is in milliseconds. "25" is 40 fps.).

For example, this is very smooth for me:

import numpy as np
import matplotlib.pyplot as plt
from matplotlib import animation

y = np.random.normal(0, 1, 10000).cumsum(axis=0)
x = np.arange(y.size)

fig, ax = plt.subplots()
line, = ax.plot([], [], '.')
ax.margins(0.05)

def init():
    line.set_data(x[:2],y[:2])
    return line,

def animate(i):
    i = min(i, x.size)
    xdata = x[:i]
    ydata = y[:i]
    line.set_data(xdata, ydata)
    ax.relim()
    ax.autoscale()
    return line,

anim = animation.FuncAnimation(fig, animate, init_func=init, interval=25)

plt.show()

I've also added ax.margins(0.05) to avoid the axes limits snapping to the next nearest "even" number and giving a "jerky" appearance.

---

However, because you're progressively plotting more and more data, the rate of change will appear to slow, simply because less of the data appears to be changing over time. Adding one point at the end of 10000 is hardly noticeable, but adding one point at the end of 10 is very noticable.

Therefore, the plot looks much more "exciting" at the beginning than at the end even though it's updating at the same rate.

This has nothing whatsoever to do with matplotlib, and is a consequence of the way you're choosing to animate your data.

To get around that, you might consider moving a "sliding window" through your data and plotting a constant number of points at a time. As an example:

import numpy as np
import matplotlib.pyplot as plt
from matplotlib import animation

y = np.random.normal(0, 1, 1000000).cumsum(axis=0)
x = np.arange(y.size) + 1

fig, ax = plt.subplots()
line, = ax.plot([], [], 'k-')
ax.margins(0.05)

def init():
    line.set_data(x[:2],y[:2])
    return line,

def animate(i):
    win = 300
    imin = min(max(0, i - win), x.size - win)
    xdata = x[imin:i]
    ydata = y[imin:i]
    line.set_data(xdata, ydata)
    ax.relim()
    ax.autoscale()
    return line,

anim = animation.FuncAnimation(fig, animate, init_func=init, interval=25)

plt.show()