Highstock: How to combine lazy loading AND dynamic data?

Question

I'm trying to combine lazy loading with dynamically updated data in Highstock. The idea is to have a large historical down-to-the-minute chart (with millions of data points) which is at the same time updated in real time (if the user scrolls to the right edge).

To prevent circular loading while loading data lazily, I need to set:

navigator : {
  adaptToUpdatedData: false
}

However, with this setting, adding a new data point to the chart does not update the navigator anymore and this prevents me to see those new data points at all (because I can't scroll past the previous right-most entry). I've tried adding those new data points to the navigator series as well but the navigator still does not get updated. I've also tried to change the max value in the xAxis of the navigator series programmatically but to no avail.

How can I make this work?

Answer 1

I had spent a few weeks solving this problem, and it is tricky. It is a very simple matter to have lazy loading (see the "1.7 million points with async loading" demo at http://www.highcharts.com/stock/demo/lazy-loading ) OR have new data points added, but both requires a bit of a hack.

First I'll detail the solution I ended up with. Do not set adaptToUpdatedData to false, simply leave it out. All the work you will do will require changes on the backend code that produces the data. Essentially, the data that is returned to Highstock will always be the full range of data that you have, however you will have a high resolution of data points for the window that is currently being viewed, but low resolution of data points for the range before and after this window. New data points are added to the end of the data as you would normally do (see the "dynamically updated data" demo at http://www.highcharts.com/stock/demo/dynamic-update )

This ensures that when new data points are added, the navigator will also be updated. However, it also prevents you from having to load the full number of data points (since the before and after ranges only contain a low resolution of data points). You will need to write code that correctly averages your data into fewer data points before returning it to the highstocks chart.

In the setExtremes handler function (which gets called when the user scrolls the navigator), I have code that makes an AJAX request for new data to add to the Highstocks chart. I also have code that uses setInterval() to fetch new data points every 10 seconds. You need a variable that tracks when the setInterval() update is happening, because if it adds new data points to the chart, this will also end up calling the setExtremes handler function, which will then make yet another AJAX request. The AJAX request in your setExtremes handler should only run if a setInterval() update isn't happening. (It is easy to have some bug that causes an infinite loop of AJAX requests for data.)

You will also need a variable in the AJAX request for data so that the back end code knows if the request is from setInterval() or from your setExtremes handler (when the user scrolled the navigator), so that it knows if it should just return the data for the requested time range as is (for setInterval() request) or do the low/high resolution averaging for the full range of data (for setExtremes handler requests).

I don't think these variables completely erase the potential for race conditions.

The problems you described are correct: You can't set adaptToUpdatedData to false and add new data points because the navigator will not update. This prevents you from scrolling the navigator to the new points. But if you don't set it, when you request the new (and smaller) range of data, the navigator will update to fit this range. The navigator and the main chart will show the same range of data, making the navigator useless.

Sebastian's answer doesn't work, unfortunately. Updating series[0] won't affect the navigator because setting adaptToUpdatedData to false disables the navigator from redrawing itself.

This low/high resolution approach is a hack, but it is the only way I found that works (after going through a half-dozen or so other ways that didn't work.) It would be great if the Highstocks team could add this feature to their library.