Big d3.js graph, canvas or server-side rendering?

Question

I'm using d3.js to visualize timeseries-data sent from my python backend (via Websocket). The usual amount of data for one graph is about 120 entries (2 hours of data, 1 entry per Minute). This is running fine, gets updated every minute.

But it should also be capable of visualizing data from a month or more (could be up to a year), also in a 1-Minute-interval. Rendering such an amount of data is too much for SVG.

I'm thinking of the following alternatives:

• Rendering it in a canvas. Is it really that much faster?
• Switching to another library like Highchart.js (Saw a demo with ~50k entries)
• Rendering the SVG/JPG/PNG on the server. Any experience on rendering d3.js server-side with e.g. phantom.js? I'd like to reuse the already written graph-models. But it could also be any other library which is able to render the data (generate the graphs with python)

What would you recommend?

Answer 1

Note that d3 supports using javascript's buffered arrays. SVG plots with thousands of time series data points has worked fine in my experience (even with multiple sources of live data streaming at 20ms updates via websockets).

For example if you pack all your data in Python; you might not need to do this in your live view as your update rate is relatively slow:

import struct
# fake data point
p = [56435367, 200, 1]
# <=little endian, d=float64 (for time), d=float64
msg_str = struct.pack('<' + 'd' * len(p), *p)
print(msg_str)
b'\x00\x00\x008\x15\xe9\x8aA\x00\x00\x00\x00\x00\x00i@\x00\x00\x00\x00\x00\x00\xf0?'

Then via your websocket that gets to javascript, where you can do something like:

this.ws.onmessage = function(e){
    // Just pump the raw bytes straight into CircularBuffer
    graph.databuffer.push(e.data);
    ...

And when you want to plot, assuming g is your reference to the D3 svg:

// Get a Float64Array containing all the values
var series_data = graph.databuffer.get_array_stream();
g.attr("d", graph.line(d3.zip(time, series_data)));

Naturally this should be even easier if you have all the data upfront. Are you perhaps plotting points instead of a single path? I found browsers struggle with plotting tens of thousands of individual circles (especially if they all move every 20ms!) but they can handle a path very easily.