Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I turn on cost controls on BigQuery?

Self managed cost controls in BigQuery are GA now! At least according to this blog post:

  • https://cloud.google.com/blog/products/data-analytics/whats-happening-bigquery-new-ingest-format-data-type-updates-ml-and-query-scheduling

Which links to these docs:

  • https://cloud.google.com/bigquery/docs/custom-quotas

Which link to this console page:

  • https://console.cloud.google.com/iam-admin/quotas

But I can't figure out how to set this up... Help?

like image 783
Felipe Hoffa Avatar asked Oct 16 '18 08:10

Felipe Hoffa


People also ask

How to estimate query cost in Google BigQuery?

BigQuery Mate is a Google Chrome extension through which you can estimate query cost within the query validator. However, this extension works only on the old user interface of BigQuery. Unlike the Google Cloud pricing calculator, this extension works even when you are querying less than 1 terabyte of data.

What are the best practices for controlling costs in BigQuery?

This page describes best practices for controlling costs in BigQuery. BigQuery has two pricing models for running queries: On-demand pricing: You pay for the number of bytes processed by each query. Flat-rate pricing: You pay for dedicated query processing capacity, measured in slots.

How do I limit the number of queries in BigQuery?

If you have multiple BigQuery projects and users, you can manage costs by requesting a custom quota that specifies a limit on the amount of query data processed per day. Creating a custom quota on query data allows you to control costs at the project-level or at the user-level.

How does BigQuery billing work?

If you create a large, multi-stage query, each time you run it, BigQuery reads all the data that is required by the query. You are billed for all the data that is read each time the query is run.


1 Answers

  • Go to the console URL (https://console.cloud.google.com/iam-admin/quotas)

  • Make sure you have the right project selected

enter image description here

  • Filter the quotas to focus on BigQuery

enter image description here

  • Check the box for "query usage per day"

enter image description here

  • Click "EDIT QUOTAS"

enter image description here

  • Set up your email, phone number.

  • Enter your daily limit in TiB (teras)

enter image description here

  • Click "Done", "Submit request".

  • Stay calm, and keep querying!

like image 134
Felipe Hoffa Avatar answered Oct 29 '22 04:10

Felipe Hoffa