Polars join on list items without explode/groupby

Question

A follow up from Polars lazyframe - add fields from other lazyframe as struct without a `collect`.

I now want to join on list items. Currently the only way I know of doing this would be to first explode the list, perform the join, do a group_by, then collect back as a list. I'm hoping there is a more concise alternative.

import polars as pl

companies = pl.DataFrame({
    "id": [1],
    "name": ["google"],
    "industry": [1001]
}).lazy()

industries = pl.DataFrame({
    "id": [1001],
    "name": ["tech"],
    "sectors": [[10011, 10012]]
}).lazy()

sectors = pl.DataFrame({
    "id": [10011, 10012],
    "name": ["software", "hardware"],
}).lazy()

The expected result:

expected = pl.DataFrame({
    "id": [1],
    "name": ["polars"],
    "industry": [{
        "name": "tech",
        "sectors": [[{"name": "software"}, {"name": "hardware"}]]
    }]
})

shape: (1, 3)
┌─────┬────────┬─────────────────────────────────────────┐
│ id  ┆ name   ┆ industry                                │
│ --- ┆ ---    ┆ ---                                     │
│ i64 ┆ str    ┆ struct[2]                               │
╞═════╪════════╪═════════════════════════════════════════╡
│ 1   ┆ polars ┆ {"tech",[[{"software"}, {"hardware"}]]} │
└─────┴────────┴─────────────────────────────────────────┘

Answer 1

There was no better way of doing it, but this feature will now be available in the next release of polars as of release 1.25.2 on the basis of this commit: feat: Enable joins on list/array dtypes #21687.