Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Plotting an empty interval for elasticsearch date histogram

I'm using an elasticsearch date histogram to group responses by count over time. The date histogram facet works great for this but if an interval doesn't have any responses that fall within in it it doesn't show up in the json. I figured the best way to combat this is to use javascript to fill in the gaps in a charting library. (ideally in highcharts but d3 or something else is possible). Months seem pretty easy to do but it get more complicated when I need to do it by week and day as well. Basically my problem is:

{ date: April: 5, count: 5 }, { date: June, count: 10 } 

needs to be more like

{ date: April: 5, count: 5 }, {date: May, count: null }, { date: June, count: 10 }
like image 281
candiman Avatar asked Nov 03 '22 21:11

candiman


1 Answers

min_doc_count=0 only creates intervals in between nonempty buckets. If you want to plot empty intervals outside your buckets (a few months ahead or behind of the start of your data), then add extended_bounds (docs).

In elasticsearch_dsl, to allow empty buckets out to two years ago, this looks like

    A(
        "date_histogram",
        field="publishedAt",
        calendar_interval="month",
        format="MMM yyyy",
        min_doc_count=0,
        extended_bounds={"min": f"{date:%b %Y}||-2y"},
    ),
like image 64
Noumenon Avatar answered Nov 09 '22 13:11

Noumenon