BigQuery allows you to create date-partitioned tables: https://cloud.google.com/bigquery/docs/creating-partitioned-tables
I'd like to be able to create views on top of date-partitioned tables and ideally still reap the performance benefits. Everything I've read suggests that this isn't possible?
Has anyone had any luck otherwise?
If a query uses a qualifying filter on the value of the partitioning column, BigQuery can scan the partitions that match the filter and skip the remaining partitions. This process is called partition pruning. Partition pruning is the mechanism BigQuery uses to eliminate unnecessary partitions from the input scan.
A partitioned view is a view defined by a UNION ALL of member tables structured in the same way, but stored separately as multiple tables in either the same instance of SQL Server or in a group of autonomous instances of SQL Server servers, called federated database servers.
Define your view to expose the partitioning pseudocolumn, like this:
SELECT *, EXTRACT(DATE FROM _PARTITIONTIME) AS date
FROM Date partitioned table;
Now if you query the view using a filter on date
, it will restrict the partitions that are read.
For anyone trying to do this with a wildcard partition date table such as Firebase or Google Analytics:
create view some.view as (
select *, _TABLE_SUFFIX as suffix from
`firebase-public-project.analytics_153293282.events_*`
)
select * from some.view WHERE suffix = '20180814'
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With