Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Weird issue: Bigquery dataset not found

One of our BQ datasets is no longer accessible via BQ Web UI and Cloud Shell.

It shows message "Not found: Dataset project:dataset" immediately upon opening the UI.

We tried a couple of bq shell commands as well:

  • bq ls: successfully lists the "missing" dataset
  • bq ls dataset: returns "BigQuery error in ls operation: Not found: Dataset project:dataset"

But we were able to query the views inside and access the contents of the dataset via PowerBI.

IAM Permission: Owner

Anyone encountering similar issue?

like image 990
Gean Faulve Avatar asked May 09 '18 06:05

Gean Faulve


People also ask

How do you handle errors in BigQuery?

BigQuery hasn't documented it yet, but you can handle any type of exception in BigQuery by creating an exception handling clause, as described in the following example: BEGIN SELECT 1/0; EXCEPTION WHEN ERROR THEN SELECT @@error. message, @@error.

How do I change the dataset location in BigQuery?

After you create the dataset, the location cannot be changed, but you can copy the dataset to a different location, or manually move (recreate) the dataset in a different location. BigQuery processes queries in the same location as the dataset that contains the tables you're querying.


1 Answers

I had a similar issue - the BigQuery client library would list the dataset when I called ListDatasets(), but attempting to call UploadCsv() with the same dataset ID would return 404 Dataset not found.

Turns out it was because I had selected 'asia-northeast1' as the Data Location when creating the dataset - it doesn't tell you when you create the dataset that this region is treated differently, but a line in the BigQuery docs says:

If your data is in a location other than the US or EU multi-regional location, you must specify the location when you perform actions such as loading data, querying data, and exporting data.

Re-creating the dataset in the US region fixed my issue. Or you could use the options in the docs above to specify the 'asia-northeast1' location everytime instead.

like image 58
cometfish Avatar answered Sep 18 '22 18:09

cometfish