using matplotlib or pyqtgraph to graph real time data

Question

I have devices connected to my serial port and I need to poll them and then display that data in a plot. I currently have this working (slowly) using matplotlib. I could have up to 64 devices connected and each device could have 20 pieces of data to update. I've set it up so that a new window can be created and a piece of data can be added to be plotted. With each additional plotting window that is opened my update rate slows considerably.
I've tried using blit animation in matplotlib, but it's not real smooth and I can see anomolies in the update. I've tried PyQtGraph, but can't find any documentation on how to use this package, and now I'm trying PyQwt, but can't get it installed (mostly because my company won't let us install a package that will handle a .gz file). Any ideas or suggestions would be greatly appreciated.

import sys
from PyQt4.QtCore import (Qt, QModelIndex, QObject, SIGNAL, SLOT, QTimer, QThread,  QSize, QString, QVariant)
from PyQt4 import QtGui

from matplotlib.figure import Figure
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from plot_toolbar import NavigationToolbar2QT as NavigationToolbar
import matplotlib.dates as md
import psutil as p
import time
import datetime as dt
import string
import ui_plotting
import pickle

try:
  _fromUtf8 = QString.fromUtf8
except AttributeError:
  _fromUtf8 = lambda s: s

class Monitor(FigureCanvas):
"""Plot widget to display real time graphs"""
  def __init__(self, timenum):
    self.timenum=timenum
    self.main_frame = QtGui.QWidget()
    self.timeTemp1 = 0
    self.timeTemp2 = 0
    self.temp = 1
    self.placeHolder = []
    self.y_max = 0
    self.y_min = 100

# initialization of the canvas
#        self.dpi = 100
#        self.fig = Figure((5.0, 4.0), dpi=self.dpi)
    self.fig = Figure()
    FigureCanvas.__init__(self, self.fig)
#        self.canvas = FigureCanvas(self.fig)
#        self.canvas.setParent(self.main_frame)
# first image setup
#        self.fig = Figure()
#        self.fig.subplots_adjust(bottom=0.5)
    self.ax = self.fig.add_subplot(111)
    self.mpl_toolbar = NavigationToolbar(self.fig.canvas, self.main_frame,False)
    self.mpl_toolbar.setFixedHeight(24)

# set specific limits for X and Y axes
#        now=dt.datetime.fromtimestamp(time.mktime(time.localtime()))       
#        self.timenum = now.strftime("%H:%M:%S.%f")
    self.timeSec = 0      
    self.x_lim = 100
    self.ax.set_xlim(0, self.x_lim)
    self.ax.set_ylim(0, 100)
    self.ax.get_xaxis().grid(True)
    self.ax.get_yaxis().grid(True)
# and disable figure-wide autoscale
    self.ax.set_autoscale_on(False)
    self.ax.set_xlabel('Time in Seconds')
# generates first "empty" plots
    self.timeb = []
    self.user = []
    self.l_user = []
    self.l_user = [[] for x in xrange(50)]
    for i in range(50):
        self.l_user[i], = self.ax.plot(0,0)


# add legend to plot
#        self.ax.legend()


def addTime(self,t1,t2):
    timeStamp = t1+"000"
#   print "timeStamp",timeStamp
    timeStamp2 = t2+"000"
    test = string.split(timeStamp,":")
    test2 = string.split(test[2],".")        
    testa = string.split(timeStamp2,":")
    testa2 = string.split(testa[2],".")

    sub1 = int(testa[0])-int(test[0])
    sub2 = int(testa[1])-int(test[1])
    sub3 = int(testa2[0])-int(test2[0])
    sub4 = int(testa2[1])-int(test2[1])

    testing = dt.timedelta(hours=sub1,minutes=sub2,seconds=sub3,microseconds=sub4)

    self.timeSec = testing.total_seconds()

def timerEvent(self, evt, timeStamp, val, lines):
    temp_min = 0
    temp_max = 0
# Add user arrays for each user_l array used, don't reuse user arrays
    if self.y_max<max(map(float, val)):
        self.y_max = max(map(float, val))
    if self.y_min>min(map(float, val)):
        self.y_min = min(map(float, val))            
#       print "val: ",val
    if lines[len(lines)-1]+1 > len(self.user):
        for k in range((lines[len(lines)-1]+1)-len(self.user)):
            self.user.append([])


# append new data to the datasets
#        print "timenum=",self.timenum
    self.addTime(self.timenum, timeStamp)
    self.timeb.append(self.timeSec)
    for j in range((lines[len(lines)-1]+1)):
        if j >49:
            break
        if j not in lines:
            del self.user[j][:]
            self.user[j].extend(self.placeHolder)
            self.user[j].append(0)
        else:
            if len(self.timeb) > (len(self.user[j])+1):
                self.user[j].extend(self.placeHolder)
            self.user[j].append(str(val[lines.index(j)])) 

    for i in range(len(lines)):
        if i>49:
            break
        self.l_user[lines[i]].set_data(self.timeb, self.user[lines[i]])
# force a redraw of the Figure

#        if self.y_max < 2:
#            self.y_max = 2
#        if self.y_min < 2:
#            self.y_min = 0 
    if self.y_min > -.1 and self.y_max < .1:            
        temp_min = -1
        temp_max = 1
    else:
        temp_min = self.y_min-(self.y_min/10)
        temp_max = self.y_max+(self.y_max/10)


    self.ax.set_ylim(temp_min, temp_max)
    if self.timeSec >= self.x_lim:
        if str(self.x_lim)[0]=='2':
            self.x_lim = self.x_lim * 2.5
        else:
            self.x_lim = self.x_lim * 2
        self.ax.set_xlim(0, self.x_lim)
#        self.fig.canvas.restore_region(self.fig.canvas)
#        self.ax.draw_artist(self.l_user[lines[0]])
#        self.fig.canvas.blit(self.ax.bbox)
    self.fig.canvas.draw()

#        self.draw()

    self.placeHolder.append(None)

class List(QtGui.QListWidget):

  def __init__(self, parent):
    super(List, self).__init__(parent)

    font = QtGui.QFont()
    font.setFamily(_fromUtf8("Century Gothic"))
    font.setPointSize(7)
    self.setFont(font)
    self.setDragDropMode(4)
    self.setAcceptDrops(True)
    self.row = []
    self.col = []
    self.disName = []
    self.lines = []
    self.counter = 0
    self.setStyleSheet("background-color:#DDDDDD")
    self.colors = ["blue", "green", "red", "deeppink", "black", "slategray", "sienna", "goldenrod", "teal", "orange", "orchid", "lightskyblue", "navy", "darkgreen", "indigo", "firebrick", "deepskyblue", "lightskyblue", "darkseagreen", "gold"]

def dragEnterEvent(self, e):
    if e.mimeData().hasFormat("application/x-qabstractitemmodeldatalist"):
#            print "currentRow : ", self.currentRow()
#            print "self.col: ", self.col
#            print "self.row: ", self.row
#            print "self.col[]: ", self.col.pop(self.currentRow())
#            print "self.row[]: ", self.row.pop(self.currentRow())

        self.col.pop(self.currentRow())
        self.row.pop(self.currentRow())
        self.disName.pop(self.currentRow())
        self.lines.pop(self.currentRow())
        self.takeItem(self.currentRow())
    if e.mimeData().hasFormat("application/pubmedrecord"):
        e.accept()
    else:
        e.ignore() 


def dropEvent(self, e):

    items = 0
    data = e.mimeData()
    bstream = data.retrieveData("application/pubmedrecord", QVariant.ByteArray)
    selected = pickle.loads(bstream.toByteArray())
    e.accept()
#        print selected
#        if self.count() != 0:
#            j = (self.lines[self.count()-1]%len(self.colors))+1

#        else:
#            j=0
    while items < len(selected):
        j=self.counter
        if j >= len(self.colors)-1:
            j = self.counter%len(self.colors)
        m = len(self.lines)
        self.lines.append(self.counter)
#            if m != 0:
#                n = self.lines[m-1]
#                self.lines.append(n+1)
#            else:
#                self.lines.append(0)
        self.col.append(str(selected[items]))
        items = items+1
        self.row.append(str(selected[items]))
        items = items+1
        self.disName.append(str(selected[items]))
        listItem = QtGui.QListWidgetItem()
        listItem.setText(str(selected[items]))

        listItem.setTextColor(QtGui.QColor(self.colors[j]))
        self.addItem(listItem)            
        items = items+1

        self.counter += 1
def dragLeaveEvent(self, event):
    event.accept()  


class PlotDlg(QtGui.QDialog):
  NextID = 0
  filename = 'Plot'
  def __init__(self,time, callback, parent=None):
    super(PlotDlg, self).__init__(parent)
    self.id = PlotDlg.NextID
    PlotDlg.NextID += 1
    self.callback = callback
    self.setWindowFlags(Qt.Window | Qt.WindowMinimizeButtonHint | Qt.WindowMaximizeButtonHint)
    self.setAttribute(Qt.WA_DeleteOnClose,True)
    self.value = []
    print "time=",time
    self.time = time
    self.dc = Monitor(self.time)
#        self.threadPool = []

    self.listWidget = List(self)
    sizePolicy = QtGui.QSizePolicy(QtGui.QSizePolicy.Fixed, QtGui.QSizePolicy.MinimumExpanding)
    sizePolicy.setHorizontalStretch(0)
    self.listWidget.setSizePolicy(sizePolicy)
    self.listWidget.setMaximumSize(QSize(150, 16777215))

    grid = QtGui.QGridLayout()
    grid.setSpacing(0)
    grid.setContentsMargins(0, 0, 0, 0)
    grid.addWidget(self.dc.mpl_toolbar,0,0,1,12)
    grid.addWidget(self.listWidget,1,1)
    grid.addWidget(self.dc,1,0)
    grid.setColumnMinimumWidth(1,110)

    self.setLayout(grid)

def update(self, clear=0):
    if clear == 1:
 now=dt.datetime.fromtimestamp(time.mktime(time.localtime()))                         
        self.dc.timenum = now.strftime("%H:%M:%S.%f") 

        self.dc.timeSec = 0
        self.dc.x_lim = 100
        self.dc.y_max = 0
        self.dc.y_min = 100            
        del self.dc.timeb[:]
        del self.dc.user[:]
        del self.dc.placeHolder[:]

#            del self.dc.l_user[:]
#            self.dc.l_user = [[] for x in xrange(50)]
#            for i in range(50):
#                self.dc.l_user[i], = self.dc.ax.plot(0,0)
        for i in range(50):
            self.dc.l_user[i].set_data(0, 0)

#            print self.dc.l_user
#            print self.dc.user

        self.dc.ax.set_xlim(0, self.dc.x_lim)
        self.dc.fig.canvas.draw()
#        print self.value
#        print str(self.time)
#        print "time:",str(self.time)
#        self.threadPool.append( GenericThread(self.dc.timerEvent,None, str(self.time), self.value, self.listWidget.lines) )
#        self.threadPool[len(self.threadPool)-1].start()

    self.dc.timerEvent(None, str(self.time), self.value, self.listWidget.lines) 

def closeEvent(self, event):
#        self.update(1)
    self.callback(self.id)
    PlotDlg.NextID -= 1

class GenericThread(QThread):
  def __init__(self, function, *args, **kwargs):
    QThread.__init__(self)
    self.function = function
    self.args = args
    self.kwargs = kwargs

  def __del__(self):
    self.wait()

  def run(self):
    self.function(*self.args,**self.kwargs)
    return 

Answer 1

The pyqtgraph website has a comparison of plotting libraries including matplotlib, chaco, and pyqwt. The summary is:

• Matplotlib is the de-facto standard plotting library, but is not built for speed.
• Chaco is built for speed but is difficult to install / deploy
• PyQwt is currently abandoned
• PyQtGraph is built for speed and easy to install

Answer 2

I've used matplotlib and PyQtGraph both extensively and for any sort of fast or 'real time' plotting I'd STRONGLY recommend PyQtGraph, (in one application I plot a data stream from an inertial sensor over a serial connection of 12 32-bit floats each coming in at 1 kHz and plot without noticeable lag.)

As previous folks have mentioned, installation of PyQtGraph is trivial, in my experience it displays and performs on both windows and linux roughly equivalently (minus window manager differences), and there's an abundance of demo code in the included examples to guide completion of almost any data plotting task.

The web documentation for PyQtGraph is admittedly less than desirable, but the source code is well commented and easy to read, couple that with well documented and diverse set of demo code and in my experience it far surpasses matplotlib in both ease of use and performance (even with the much more extensive online documentation for matplotlib).

Answer 3

I would suggest Chaco "... a package for building interactive and custom 2-D plots and visualizations." It can be integrated in Qt apps, though you can probably get higher frame rates from PyQwt.

I've actually used it to write an "app" (that's too big a word: it's not very fancy and it all fits in ~200 LOC) that gets data from a serial port and draws it (20 lines at over 20 fps, 50 at 15 fps, at full screen in my laptop).

Chaco documentation or online help weren't as comprehensive as matplotlib's, but I guess it will have improved and at any rate it was enough for me.

As a general advice, avoid drawing everything at every frame, ie., use the .set_data methods in both matplotlib and chaco. Also, here in stackoverflow there are some questions about making matplotlib faster.