Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Firebase count group by

Tags:

firebase

Does Firebase supports grouped counting?
I would like to get counting for specific key grouped by the value.

Example of my data structure:

"playbackPosition" : {
    "-JZ2c7v-imQXaEx2c4Cs" : {
        "data" : "0"
    },
    "-JZ2cDta73UGAgwIOejL" : {
        "data" : "25"
    },
    "-JZ2cJutDZu-7quLpLI2" : {
        "data" : "50"
    },
    "-JZ2cO-Me0nGK5XLTwd-" : {
        "data" : "75"
    },
    "-JZ2cSh-XnstYxbE_Zad" : {
        "data" : "100"
    },
    "-JZ2cWxya0-kmTPCze4u" : {
        "data" : "0"
    },
    "-JZ2c_wX-ODv43TKXkNG" : {
        "data" : "25"
    }
}

Required results based on the data key :

0 => 2  
25 => 2  
50 => 1  
75 => 1  
100 => 1

And of course I must consider that it will have thousands of children's, not only 7...

Thanks ahead!

EDIT
Deeper explanation of the app and the problem we want to solve.

We have video scripts which runs on different websites, each video session (a user session) sends events and data, you can see an example here, check the Network tab - https://secure.bwebi.co/videostir/regular.html

Our goal is to collect this data and create an analytics real time dashboard with few charts and graphs.

You can see our current data structure here

Example for graphs we need:
Completion rate
General - Bar graph showing overall number of views per clip duration pre defined periods.
Filters - date (start/end), unique/all, All urls / specific urls (embed on), All configurations / specific configurations/ ignore silent.
X axis - groups 0-25, 25-50,50-75,75-99, 100
Y axis - number of views

Views per day (with completion rate)
General - Multi lines graph showing number of views per day per duration periods.
Filters - date (start/end), unique/all, All urls / specific urls (embed on), All configurations / specific configurations / ignore silent.
X axis - Time in days
Y axis - Number of views
Lines for:
Total daily views
Daily views with 100% duration
Daily views with 75-99% duration
Daily views with 50-75% duration
Daily views with 25-50% duration
Daily views with 0-25% duration

Hope it's more clear now!

like image 425
Chen Alon Avatar asked Oct 12 '14 23:10

Chen Alon


1 Answers

Group by is a SQL function. The reason SQL can't do real-time data is because this sort of method does not scale. Mongo provides similar functionality, but once again, it doesn't scale. You may notice a pattern here of why Firebase does not provide this sort of query function.

It would be extremely helpful if you provided some context of what you're actually attempting to accomplish here, what the rules of the app are, and what approaches you've ruled out, rather than just your presupposed solution of group by. There are probably other, possibly better, alternatives. See the XY problem.

Here are a couple generic alternatives derived by making sweeping assumptions about your use case.

Store the totals

This is the most scalable solution. Store your data as follows:

/playbacks/$id/<playback data>
/group_totals/$group/<count>

When writing to playbacks, also update the count for the appropriate group:

var fb = new Firebase(URL);
function addPlayback(rec) {
   var ref = fb.child('playbacks').push(rec, function(err) {
      if( err )  throw err;
      incrementCount(rec.data);
   });
}

function incrementCount(count) {
   fb.child('group_totals/' + count).transaction(function(currentVal) {
     return (currentVal||0)+1;
   });
}

Now when you want to get the total for a group, you can simply look up the value at group_totals/$group. Similarly, you can store ids for records that belong to each group and utilize that index to grab only the records for a given group.

Use priorities to fetch and group

A simpler approach would be to give each record a priority based on the group/data value.

var fb = new Firebase(URL);
function addPlayback(rec) {
   rec['.priority'] = rec.data;
   var ref = fb.child('playbacks').push(rec, function(err) {
      if( err )  throw err;
   });
}

Now to grab a set of records for a given group:

var fb = new Firebase(URL);
function getGroup(groupValue, callback) {
   fb.child('playbackPosition').startAt(groupValue).endAt(groupValue).once('value', callback);
}

function logGroupCount(groupValue, callback) {
   getGroup(groupValue, function(snap) {
       console.log('there are ' + snap.numChildren() + ' items in group ' +groupValue);
   });
}
like image 164
Kato Avatar answered Oct 06 '22 08:10

Kato