Count cumulative total in Postgresql

Question

I am using count and group by to get the number of subscribers registered each day:

  SELECT created_at, COUNT(email)       FROM subscriptions  GROUP BY created at; 

Result:

created_at  count ----------------- 04-04-2011  100 05-04-2011   50 06-04-2011   50 07-04-2011  300 

I want to get the cumulative total of subscribers every day instead. How do I get this?

created_at  count ----------------- 04-04-2011  100 05-04-2011  150 06-04-2011  200 07-04-2011  500 

Answer 1

With larger datasets, window functions are the most efficient way to perform these kinds of queries -- the table will be scanned only once, instead of once for each date, like a self-join would do. It also looks a lot simpler. :) PostgreSQL 8.4 and up have support for window functions.

This is what it looks like:

SELECT created_at, sum(count(email)) OVER (ORDER BY created_at) FROM subscriptions GROUP BY created_at; 

Here OVER creates the window; ORDER BY created_at means that it has to sum up the counts in created_at order.

---

Edit: If you want to remove duplicate emails within a single day, you can use sum(count(distinct email)). Unfortunately this won't remove duplicates that cross different dates.

If you want to remove all duplicates, I think the easiest is to use a subquery and DISTINCT ON. This will attribute emails to their earliest date (because I'm sorting by created_at in ascending order, it'll choose the earliest one):

SELECT created_at, sum(count(email)) OVER (ORDER BY created_at) FROM (     SELECT DISTINCT ON (email) created_at, email     FROM subscriptions ORDER BY email, created_at ) AS subq GROUP BY created_at; 

If you create an index on (email, created_at), this query shouldn't be too slow either.

---

(If you want to test, this is how I created the sample dataset)

create table subscriptions as    select date '2000-04-04' + (i/10000)::int as created_at,           '[email protected]' || (i%700000)::text as email    from generate_series(1,1000000) i; create index on subscriptions (email, created_at); 

Answer 2

Use:

SELECT a.created_at,        (SELECT COUNT(b.email)           FROM SUBSCRIPTIONS b          WHERE b.created_at <= a.created_at) AS count   FROM SUBSCRIPTIONS a