Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What to try to get BigQuery to CAST BYTES to STRING?

BigQuery Standard SQL documentation suggests that BYTE fields can be coerced into STRINGS.

We have a byte field that is the result of SHA256 hashing a field using BigQuery itself.

We now want to coerce it to a STRING, yet when we run "CAST(field_name to STRING)" we get an error:

Query Failed Error: Invalid cast of bytes to UTF8 string

What is preventing us from getting a string from this byte field? Is it surmountable? If so, what is the solution?

like image 660
Praxiteles Avatar asked Apr 04 '18 21:04

Praxiteles


People also ask

How do I CAST a BigQuery?

Cast syntax is used in a query to indicate that the result type of an expression should be converted to some other type. When using CAST , a query can fail if BigQuery is unable to perform the cast. If you want to protect your queries from these types of errors, you can use SAFE_CAST.

How do I CAST an int to a date in BigQuery?

Cast(cast(20190101 AS STRING) AS date) FORMAT_DATE("%F",CAST(20190101 AS STRING))

How do I extract a string between two characters in BigQuery?

To get the substring between two characters, we will use the REGEXP_SUBSTR() function. This function returns the value of the substring that matches the regular expression, or NULL if no match is found. Note: REGEXP_SUBST() is the regular expression equivalent of SUBSTR() .

What does the BigQuery lag function do?

LAG. Returns the value of the value_expression on a preceding row. Changing the offset value changes which preceding row is returned; the default value is 1 , indicating the previous row in the window frame. An error occurs if offset is NULL or a negative value.

How do I combine strings in BigQuery?

The BigQuery CONCAT function allows you to combine (concatenate) one more values into a single result. Alternatively, you can use the concatenation operator || to achieve the same output.


2 Answers

Below example should show you an idea

#standardSQL
WITH t AS (
  SELECT SHA256('abc') x
)
SELECT x, TO_BASE64(x)
FROM t 

in short - you can use TO_BASE64() for this

like image 197
Mikhail Berlyant Avatar answered Sep 21 '22 19:09

Mikhail Berlyant


If you want to see the "traditional" representation of the hash in String, you have to use TO_HEX() function.

WITH table AS (
 SELECT SHA256('abc') as bytes_field
)
SELECT bytes_field, TO_HEX(bytes_field) as string_field
FROM table

By default in the UI, BigQuery shows you the base64 representation but if you want to compare it with other sha256 function from other language, for example, you have to use TO_HEX()

like image 34
Javier Montón Avatar answered Sep 21 '22 19:09

Javier Montón