Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to filter empty groups from a reduction?

Tags:

crossfilter

It appears to me that Crossfilter never excludes a group from the results of a reduction, even if the applied filters have excluded all the rows in that group. Groups that have had all of their rows filtered out simply return an aggregate value of 0 (or whatever reduceInitial returns).

The problem with this is that it makes it impossible to distinguish between groups that contain no rows and groups that do contain rows but just legitimately aggregate to a value of 0. Basically, there's no way (that I can see) to distinguish between a null value and a 0 aggregation.

Does anybody know of a built-in Crossfilter technique for achieving this? I did come up with a way to do this with my own custom reduceInitial/reduceAdd/reduceRemove method but it wasn't totally straight forward and it seemed to me that this is behavior that might/should be more native to Crossfilter's filtering semantics. So I'm wondering if there's a canonical way to achieve this.

I'll post my technique as an answer if it turns out that there is no built-in way to do this.

like image 906
Scott Cameron Avatar asked Jan 25 '14 15:01

Scott Cameron


1 Answers

A simple way to accomplish this is to have both count and total be reduce attributes:

var dimGroup = dim.group().reduce(reduceAdd, reduceRemove, reduceInitial);

function reduceAdd(p, v) {
  ++p.count;
  p.total += v.value;
  return p;
}

function reduceRemove(p, v) {
  --p.count;
  p.total -= v.value;
  return p;
}

function reduceInitial() {
  return {count: 0, total: 0};
}

Empty groups will have zero counts, so retrieving only non-empty groups is easy:

dimGroup.top(Infinity).filter(function(d) { return d.value.count > 0; });
like image 172
davidshinn Avatar answered Oct 18 '22 14:10

davidshinn