Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

"Exceeded quota: too many free query bytes scanned for this project" in Google BigQuery

I'm getting the above error since yesterday, after trying out Google BigQuery with JasperReports Server. I couldn't find any info on this, since the first 100GB should be free and the BigQuery Access Report is saying:

BigQuery Resource Usage for API Project Data Analysis Jan 31, 2013 – Feb 27, 2013

Data Processed Per Day 0 MB Peak 0.00 MB Average

I haven't reached the free query/user/day limit either: API Per-User Limit Used Billable Limit Courtesy Limit
BigQuery API 5.0 requests/second/user
0% 200,000 requests/day 10,000 requests/day

and

Traffic Reports for API Project Total requests 935 Requests/day 856 peak 33.39 average

Start Date Jan 31, 2013

Sample Period 28 days

I'm trying to use the natality sample data in a project for my university and it's pretty urgent. It was working fine for a day or so. The queries are made by the JasperSoft BigQuery Connector (http://code.google.com/p/jaspersoft-bigquery-datasource/) using an Ad-hoc-View (based on a JasperReport as Topic).

I am using the following statement:

SELECT
    STRING($P!{dimension1}) as dimension_1,
    STRING($P!{dimension2}) as dimension_2,
    SUM(record_weight)  as count_group,
    AVG(weight_pounds)  as avg_weight_pounds,
    AVG(apgar_1min) as apgar_1min,
    AVG(apgar_5min) as apgar_5min
FROM publicdata:samples.natality
WHERE
    year >= $P!{year_first}
    AND year <= $P!{year_last}
GROUP BY dimension_1, dimension_2
ORDER BY dimension_1, dimension_2

Any help would be highly appreciated! Thanks in advance!

like image 749
MisterXtm Avatar asked Feb 27 '13 20:02

MisterXtm


People also ask

How much query capacity is in the free BigQuery quota?

Your project can run up to four concurrent queries against a Bigtable external data source. You can run up to ten concurrent queries with remote functions per project. Your project can run up to 1,000 concurrent multi-statement queries.

How much free storage per month does BigQuery provide?

The first 10 GB of storage per month is free. If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply. Note: If you plan to store more than 1 PB of data in BigQuery, contact your sales representative for pricing.

What is the maximum column limit for the BigQuery?

The limit is 10,000 columns.


2 Answers

Summarising the comments above and below for anyone else's benefit:

  • The quota for your usage doesn't show up in the developer console if you don't enable billing.
  • Looking at the reports page will show 0GB used if you don't enable billing.
  • Unless you enable billing, you have a limit of 100GB to query per month.
  • Enabling billing will enable you to query more than 100GB of data.
  • The BigQuery team are working on a fix for the reports page not including queries on data when you haven't enabled billing
like image 64
edhgoose Avatar answered Nov 15 '22 08:11

edhgoose


Even in 2021 still a relevant question. Now with PowerBi connected to BigQuery (Direct Queries). It shows an error message, in my case with code 31780: quota exceeded free query bytes scanned.

Here's the problem

  • When the data is connected BQ seems to not connect to the correct project.

Here's how to solve it:

  • Click the table with the BQ connection
  • Click on the transform data (it's on the home tab)
  • Within transform data go to the view tab
  • Click on advanced editor
  • You'll see this line:
    Source = GoogleBigQuery.Database(null),
  • Change it to:
    Source = GoogleBigQuery.Database([BillingProject="project-id"]),

Note: not the project name, the project ID

Do this for all tables that fetch data from BQ.

source: https://docs.microsoft.com/en-gb/power-bi/connect-data/desktop-connect-bigquery#considerations-and-limitations

like image 32
Michel K Avatar answered Nov 15 '22 08:11

Michel K