Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to query a BYTES field in BigQuery using SQL?

We have SHA256 encrypted a field in our table on Google BigQuery which left the result type as BYTES.

We have tried writing various matching field queries but none are apparently correct.

SELECT * WHERE
field LIKE '16D6M7PN3w7Cn8mJyrmrUSZY9ummMf5QCGEMuiSmSlw=' 

SELECT ...
field = '16D6M7PN3w7Cn8mJyrmrUSZY9ummMf5QCGEMuiSmSlw='

SELECT ...
field = 16D6M7PN3w7Cn8mJyrmrUSZY9ummMf5QCGEMuiSmSlw=

How does one write a SQL query in BigQuery to query BYTES fields?


Update: Now we have the query running using Gordon's answer below but it is returning zero results even though the byte code is an exact match for what BigQuery is showing. Is there something more we need to do to match bytes?

like image 959
Praxiteles Avatar asked Feb 04 '17 13:02

Praxiteles


People also ask

What is the best way to query BigQuery data?

Standard SQL is preferred for querying data stored in BigQuery because it’s compliant with the ANSI SQL 2011 standard. It has other advantages over legacy SQL, such as automatic predicate push down for JOIN operations and support for correlated subqueries.

How to create an array of arrays in BigQuery?

BigQuery does not support building arrays of arrays directly. Instead, you must create an array of structs, with each struct containing a field of type ARRAY. To illustrate this, consider the following points table: Now, let's say you wanted to create an array consisting of each point in the points table.

What is the difference between null and empty array in BigQuery?

BigQuery translates a NULL ARRAY into an empty ARRAY in the query result, although inside the query, NULL and empty ARRAYs are two distinct values. BigQuery raises an error if the query result has an ARRAY which contains NULL elements, although such an ARRAY can be used inside the query. For example, this works:

What are the functions supported by BigQuery?

In addition to aggregate and analytic functions, BigQuery also supports functions and operators such as string manipulation, date/time, mathematical functions, JSON extract and more, as shown. Refer BigQuery SQL function reference for the complete list.


Video Answer


1 Answers

I think the UI is showing the base 64-escaped contents of the bytes columns, since they can't be rendered as UTF-8. You'll need to use e.g.

SELECT *
FROM T
WHERE field = FROM_BASE64('16D6M7PN3w7Cn8mJyrmrUSZY9ummMf5QCGEMuiSmSlw=');
like image 192
Elliott Brossard Avatar answered Sep 18 '22 05:09

Elliott Brossard