Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Populating a table with all dates in a given range in Google BigQuery

Is there any convenient way to populate a table with all dates in a given range in Google BigQuery? What I need are all dates from 2015-06-01 till CURRENT_DATE(), so something like this:

+------------+
| date       |
+------------+
| 2015-06-01 |
| 2015-06-02 |
| 2015-06-03 |
| ...        |
| 2016-07-11 |
+------------+

Optimally, the next step would be to also get all weeks between the two dates, i.e.:

+---------+
| week    |
+---------+
| 2015-23 |
| 2015-24 |
| 2015-25 |
| ...     |
| 2016-28 |
+---------+

I've been fiddling around with the following answers I found, but I can't get them to work, mostly because core functions aren't supported and I can't find proper ways to replace them.

Easiest way to populate a temp table with dates between and including 2 date parameters

Generate Dates between date ranges

Your help is very much appreciated!

Best, Max

like image 293
Max Avatar asked Jul 11 '16 11:07

Max


People also ask

How do you create a date range in BigQuery?

BigQuery SQL is offering one neat array function GENERATE_DATE_ARAY , where you can specify the following inputs [1]: start_date — must be a DATE. end_date — must be a DATE. INT64_expr — a parameter which determines the increment used to generate dates; the default value for this parameter is one day.

WHAT IS WITH clause in BigQuery?

The WITH clause contains one or more common table expressions (CTEs). Each CTE binds the results of a subquery to a table name, which can be used elsewhere in the same query expression. BigQuery does not materialize the results of non-recursive CTEs within the WITH clause.


2 Answers

Mikhail's answer works for BigQuery's legacy sql syntax perfectly. This solution is a slightly easier one if you're using the standard SQL syntax.

BigQuery standard SQL syntax actually has a built in function, GENERATE_DATE_ARRAY for creating an array from a date range. It takes a start date, end date and INTERVAL. For example:

SELECT day
FROM UNNEST(
    GENERATE_DATE_ARRAY(DATE('2015-06-01'), CURRENT_DATE(), INTERVAL 1 DAY)
) AS day

If you wanted the week and year you could use

SELECT EXTRACT(YEAR FROM day), EXTRACT(WEEK FROM day)
FROM UNNEST(
    GENERATE_DATE_ARRAY(DATE('2015-06-01'), CURRENT_DATE(), INTERVAL 1 WEEK)
) AS day
like image 144
Tim Martin Avatar answered Oct 19 '22 17:10

Tim Martin


all dates from 2015-06-01 till CURRENT_DATE()

SELECT DATE(DATE_ADD(TIMESTAMP("2015-06-01"), pos - 1, "DAY")) AS DAY
FROM (
     SELECT ROW_NUMBER() OVER() AS pos, *
     FROM (FLATTEN((
     SELECT SPLIT(RPAD('', 1 + DATEDIFF(TIMESTAMP(CURRENT_DATE()), TIMESTAMP("2015-06-01")), '.'),'') AS h
     FROM (SELECT NULL)),h
)))

all weeks between the two dates

SELECT YEAR(DAY) AS y, WEEK(DAY) AS w
FROM (
  SELECT DATE(DATE_ADD(TIMESTAMP("2015-06-01"), pos - 1, "DAY")) AS DAY
  FROM (
       SELECT ROW_NUMBER() OVER() AS pos, *
       FROM (FLATTEN((
       SELECT SPLIT(RPAD('', 1 + DATEDIFF(TIMESTAMP(CURRENT_DATE()), TIMESTAMP("2015-06-01")), '.'),'') AS h
       FROM (SELECT NULL)),h
  )))
)
GROUP BY y, w
like image 32
Mikhail Berlyant Avatar answered Oct 19 '22 18:10

Mikhail Berlyant