Loading several large dataset in Highcharts

Question

In Highcharts, there are demos and articles on loading large data sets into Highcharts.

We are aware that none of today's browsers are capable of doing that if the data get too large. Refer to this article, it suggests to use aggregation and separation of tables to load data into HighCharts, but I just don't think the solution fits my need.

I use mysql to store my data and have lots of tables. The data to be plotted on Highcharts is time series based, which means the data is once a day.

I need to load the chart with several series set of this data (base on criterion), and the data cannot be aggregated to monthly as it needs to be displayed as in daily mode only. On top of this I have to set the first view of the chart as All.

The reason for not to aggregate the data because the user wants to see the daily changes. Sometime there are irregular data and aggregation is not the best options for this. The data to be loaded is from year 1980

What is the best strategy or technique to load these data onto HighCharts?

Answer 1

So, you're saying that you have 35year of data on daily basis. For me it's < 13k of points. And then you have chart, for example 600px wide. That means you have 20points on 1pixel. And you saying that aggregation is bad, because user need to see true data. Let me ask, how user will see that points, when will have 20 of them in 1px? Maybe I'm getting old, but I won't be able to see that points really good, just some 'fat path'. Compare these examples: http://jsfiddle.net/Fusher/4ytzuv7o/1/

Just disabled dataGrouping in a second example:

dataGrouping: {
    enabled: false
}

When you zoom-in in a first example, you will get true data anyway.

If you really can't use dataGrouping, then how about another solution? Disable navigator and 'All' button, to user will be able to zoom only at specified ranges, like this: http://jsfiddle.net/Fusher/4ytzuv7o/3/

$('#container').highcharts('StockChart', {
    rangeSelector: {
        inputEnabled: false,
        buttons: [{
            type: 'month',
            count: 1,
            text: '1m'
        }, {
            type: 'month',
            count: 3,
            text: '3m'
        }, {
            type: 'month',
            count: 6,
            text: '6m'
        }, {
            type: 'ytd',
            text: 'YTD'
        }, {
            type: 'year',
            count: 1,
            text: '1y'
        }],
        selected: 0
    },
    navigator: {
        enabled: false   
    },
    series: [{
        name: 'AAPL',
        data: data,
        dataGrouping: {
            enabled: false   
        }
    }]
});

Answer 2

Have you tried loading the daily data? If you have a data point for every day over the past 45 years then it's just 16,000 points. Regardless, I would start by first making sure loading everything is too much for highcharts and if so, consider a dynamic aggregation script where zooming past a threshold, say 5 years, triggers the data set to the aggregate set per week.

This is my potential solution:

Load the chart with the full resolution data set, the per week aggregate set, and maybe per month set. You then set an event listener on the number of points on the current view. When it reaches a certain threshold, you redraw to aggregate.

You won't really lose the chance to see drastic events on a specific day since aggregating 7 days will still show a significant enough peak. The user can then zoom in and see the more detailed data. Forty five year's worth of daily data reduces down to a few thousand points.