Path transition with gap - how do to that?

Question

I draw some line chart (path) that has some "gaps". I use a trick between the gaps, then the interpolation doesn't appears between the gaps I have on my data.

Example:

pos, value
1,10
2,15
3,20
8,5
9,6
10,20

As you can see, I have a gap between position 3 and 8. Here what I do is to create a position 4 with value 0, and a position 7 with value 0. Then, whenever I plot my line chart I get the correct chart with my gap.

Now, I need to make a transition on this line chart. Given I have this "gap hack", my transition is not beatiful :( It kind of move lines in a way that I don't like.

What should I do ? Is it possible to make a transition in this case without the need of having a lot of 0 values between my gaps ?

I don't want to add 0 values for positions I don't need to represent, otherwise my database will increase a LOT and my visualization would be slow.

Any insight is highly appreciated.

Answer 1

The first thing you need to do in you update function is to create a new data array the fills in the missing values for your position variable, while leaving the value variable undefined. Something like:

var data = [/* Your original data table */];
       //results of d3.tsv or whatever you use

var posRange = d3.max(data, function(d){return d.pos;});  
    //get the max and min pos values as a two-element array
var dataWithGaps = new Array(posRange[1] + 1 - posRange[0] );
    //i.e., length of the array is the difference in position, 
    //plus 1 to include the last position 
    //(eg for positions 2-5 you need 4 elements 2,3,4,5, which is 5+1-2

var j=dataWithGaps.length;
while (j--){ //count down the length of the new array
    dataWithGaps[j] = {pos:posRange[0]+j, value:NaN}; 
        //create a valid position and undefined value
}
j=data.length;
while (j--){//scan through the data and copy over
    d = data[j];
    dataWithGaps[d.pos-posRange[0]].value = d.value; 
      //find the correct index in the new array, and set it's value
}

If your positions are integers counting up from zero or one, then this code can be simplified considerably, just using the index of the array as your position values. If your positions are timestamps or something else, the code gets more complicated.

Either way, when you update your data, be sure to check whether the range has changed, and therefore whether you need to pad the start or end of your dataWithGaps array (with push or unshift) with more position values. Then repeat the last two loops.

Now to graphing...

The following examples all start with an array of numbers containing NaN (not-a-number) values as gaps, where the index of the array is used as the x-coordinate.

First example, maybe you were already doing this, is to use the d3 path-generator function's .defined(function(d){}) method. This allows you to indicate which data points should be undefined, and d3 automatically breaks your line into segments around the undefined points.

Ta-Dah! http://fiddle.jshell.net/9jTk4/2/)

The magic code is the last statement in this chain:

var drawLine = d3.svg.line()
    .x(function(d,i) { return xScale(i); })
    .y(function(d) { return yScale(d); })
    .defined(function(d) {return !isNaN(d)});
    //the line should be only be defined at points where the data value is not NaN
    //use !isNaN(d.value) if your data values are objects

Note that the data-points are made using SVG line markers so that you can see points even if they don't connect to anything because their neighbours are undefined.

(Confession time: I only discovered this after I spent a few hours coming up with a custom solution. Luckily, my extra time wasn't wasted because there is a flaw with the d3 default approach...)

The problem is when you add a transition.

Dum-da-DUM (Ominous organ chords) http://fiddle.jshell.net/9jTk4/1/

D3 can't doesn't know where to start the new line segment for the transition. If you're only adding one new point in between two existing points, it transitions smoothly from the previous point. However, if if you've got multiple points adding in to a gap it starts them out at the position of the final element in the array -- which might work great when you're adding on to the end of the line, but looks pretty ugly if you're filling in gaps.

The transition could probably be fixed with a custom tween function, but that's not going to be straightforward to write from scratch. (Though, if someone does want to create one, it would probably be worth making it a pull request to be added to the d3 standard library.)

In the meantime, take a look at this alternative approach I created (before I read the d3 API carefully enough to discover that they already account for gaps).

Ta-Dah! and AlaKaZAM!!! http://fiddle.jshell.net/pwmA3/3/

It looks very similar to the last version, except with smooth transitions, but it is actually very different in construction. Instead of using a single path to represent the entire line, it uses a series of svg:line elements, each of which draws the path up to the data point from the previous point (if it existed).

If a data point doesn't exist, it's plotting position is interpolated, and it's class is set to trigger CSS that makes the line disappear, and then transitions it in smoothly when it's given valid data. Line-markers again show all the valid data points, even if there is nothing to link them to. Here's a version where I've added a "question mark" line marker for the points with no data, showing how their positions were interpolated:
http://fiddle.jshell.net/pwmA3/4/

One thing to note: using line segments for each data point instead of a path means a lot more DOM objects for the browser to keep track of. If you have many hundreds of data points, this could slow things down, as could all the extra computation for interpolating the missing points. If that becomes a problem, I'd recommend using the d3 line graph method with the defined() option (as in the first example), and just skip the transition.