BigQuery fails to save view that uses functions

Question

We're using BigQuery with their new dialect of "standard" SQL. the new SQL supports inline functions written in SQL instead of JS, so we created a function to handle date conversion.

CREATE TEMPORARY FUNCTION
  STR_TO_TIMESTAMP(str STRING)
  RETURNS TIMESTAMP AS (PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E*SZ', str));

It must be a temporary function as Google returns Error: Only temporary functions are currently supported; use CREATE TEMPORARY FUNCTION if you try a permanent function.

If you try to save a view with a query that uses the function inline - you get the following error: Failed to save view. No support for CREATE TEMPORARY FUNCTION statements inside views. If you try to outsmart it, and remove the function (hoping to add it during query time), you'll receive this error Failed to save view. Function not found: STR_TO_TIMESTAMP at [4:7].

Any suggestions on how to address this? We have more complex functions than the example shown.

Answer 1

Since the issue was marked as resolved, BigQuery now supports permanents registration of UDFs. In order to use your UDF in a view, you'll need to first create it.

CREATE OR REPLACE FUNCTION `ACCOUNT-NAME11111.test.STR_TO_TIMESTAMP`
    (str STRING) 
    RETURNS TIMESTAMP AS (PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E*SZ', str));

• Note that you must use a backtick for the function's name.
• There's no TEMPORARY in the statement, as the function will be globally registered and persisted.
• Due to the way BigQuery handles namespaces, you must include both the project name and the dataset name (test) in the function's name.

Once it's created and working successfully, you can use it a view.

create view test.test_view as
select `ACCOUNT-NAME11111.test.STR_TO_TIMESTAMP`('2015-02-10T13:00:00Z') as ts

You can then query you view directly without explicitly specifying the UDF anywhere.

select * from test.test_view

Answer 2

Saving a view created with a temp function is still not supported, but what you can do is plan the SQL-query (already rolled out for the latest UI), and then save it as a table. This worked for me, but I guess it depends on the query parameters you want.

##standardSQL
## JS in SQL to extract multiple h.CDs at the same time. 
CREATE TEMPORARY FUNCTION getCustomDimension(cd ARRAY<STRUCT< index INT64, 
value STRING>>, index INT64)
RETURNS STRING
LANGUAGE js AS """
     for(var i = 0; i < cd.length; i++) {
     var item = cd[i];
     if(item.index == index) {
         return item.value
  }
}
return '';
""";

SELECT DISTINCT h.page.pagePath, getcustomDimension(h.customDimensions,20), fullVisitorId,h.page.pagePathLevel1, h.page.pagePathLevel2, h.page.pagePathLevel3, getcustomDimension(h.customDimensions,3)
FROM
`XXX.ga_sessions_*`,
UNNEST(hits) AS h
WHERE
    ### rolling timeframe
    _TABLE_SUFFIX = FORMAT_DATE('%Y%m%d',DATE_SUB(CURRENT_DATE(),INTERVAL YY DAY))
    AND h.type='PAGE'

Credit for the solution goes to https://medium.com/@JustinCarmony/strategies-for-easier-google-analytics-bigquery-analysis-custom-dimensions-cad8afe7a153

Answer 3

As per the documentation https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_function_statement , the functionality is still in Beta phase but is doable. The functions can be viewed in the same dataset it was created and the view can be created. Please share if that worked fine for you or if you have any findings which would be helpful for others.