How to plot large data vectors accurately at all zoom levels in real time?

Question

I have large data sets (10 Hz data, so 864k points per 24 Hours) which I need to plot in real time. The idea is the user can zoom and pan into highly detailed scatter plots.

The data is not very continuous and there are spikes. Since the data set is so large, I can't plot every point each time the plot refreshes.

But I also can't just plot every nth point or else I will miss major features like large but short spikes.

Matlab does it right. You can give it a 864k vector full of zeros and just set any one point to 1 and it will plot correctly in real-time with zooms and pans.

How does Matlab do it?

My target system is Java, so I would be generating views of this plot in Swing/Java2D.

Answer 1

You should try the file from MATLAB Central:

https://mathworks.com/matlabcentral/fileexchange/15850-dsplot-downsampled-plot

From the author:

This version of "plot" will allow you to visualize data that has very large number of elements. Plotting large data set makes your graphics sluggish, but most times you don't need all of the information displayed in the plot. Your screen only has so many pixels, and your eyes won't be able to detect any information not captured on the screen.

This function will downsample the data and plot only a subset of the data, thus improving the memory requirement. When the plot is zoomed in, more information gets displayed. Some work is done to make sure that outliers are captured.

Syntax:

dsplot(x, y)  
dsplot(y)  
dsplot(x, y, numpoints)  

Example:

x =linspace(0, 2*pi, 1000000);  
y1=sin(x)+.02*cos(200*x)+0.001*sin(2000*x)+0.0001*cos(20000*x);  
dsplot(x,y1);