Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

BigQuery error: Cannot partition on repeated field

I have two tables table1 (complex one with repeated/record columns) and table2 (fairly simple). I am trying to create a new table with all columns from table1 with one column from table2 using the following query:

select t1.id, t1.experience.desc, t1.experience.organization.*, t1.experience.department, t2.field2 as t1.experience.organization.newfield, t1.family_name
from [so_public.table1] as t1 left join each [so_public.table2] as t2
on t1.experience.organization.name = t2.field1

I get an error Cannot partition on repeated field as shown in the image below. The schemas of the two tables are also shown in their respective images.

Is there a general rule of thumb here when one wants to merge data from two tables? Is what I am trying to do at all possible?

The actual tables are much more complex. I am only showing enough context that reproduces the issue.

Query with errorTable1 schemaTable2 schema

like image 570
wpfwannabe Avatar asked May 26 '15 18:05

wpfwannabe


1 Answers

With the public sample you published on edit, a working query:

select t1.id, t1.experience.desc, t1.experience.department, t1.experience.organization.*, t2.field2 as t1.experience.organization.newfield, t1.family_name
from FLATTEN(FLATTEN([earnest-stock-91916:so_public.table1], experience.organization), experience) as t1 left join each [earnest-stock-91916:so_public.table2] as t2
on t1.experience.organization.name = t2.field1;

I was able to FLATTEN the data (had to apply it twice), but not to recover the original structure - joining against one of the sub-rows is harder.

I see what you want to do is enrich some of the sub-rows?

like image 199
Felipe Hoffa Avatar answered Oct 21 '22 14:10

Felipe Hoffa