Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Horizontal reduction operation on Array columns

An example of horizontal operation is pl.any_horizontal which can be apply across the pl.Boolean columns, i.e.

import polars as pl

df = pl.DataFrame(
    {
        "a": pl.Series(
            "a", 
            [False, False],
            dtype=pl.Boolean,
        ),
        "b": pl.Series(
            "b", 
            [True, False],
            dtype=pl.Boolean,
        ),
    }
)

df.with_columns(pl.any_horizontal("a", "b"))
shape: (2, 3)
┌───────┬───────┬───────┐
│ a     ┆ b     ┆ any   │
│ ---   ┆ ---   ┆ ---   │
│ bool  ┆ bool  ┆ bool  │
╞═══════╪═══════╪═══════╡
│ false ┆ true  ┆ true  │
│ false ┆ false ┆ false │
└───────┴───────┴───────┘

Is there a way to apply the same operation across columns containing pl.Array data types?

df = pl.DataFrame(
    {
        "a": pl.Series(
            "a", 
            [[False, False], [False, True]],
            dtype=pl.Array(shape=2, inner=pl.Boolean),
        ),
        "b": pl.Series(
            "b", 
            [[True, False], [True, True]],
            dtype=pl.Array(shape=2, inner=pl.Boolean),
        ),
    }
)

df.with_columns(pl.any_horizontal("a", "b"))
# InvalidOperationError: cannot cast Array type (inner: 'Boolean', to: 'Boolean')

Desired result:

shape: (2, 3)
┌────────────────┬────────────────┬────────────────┐
│ a              ┆ b              ┆ any            ┆
│ ---            ┆ ---            ┆ ---            ┆
│ array[bool, 2] ┆ array[bool, 2] ┆ array[bool, 2] ┆
╞════════════════╪════════════════╪════════════════╡
│ [false, false] ┆ [true, false]  ┆ [true, false]  │
│ [false, true]  ┆ [true, true]   ┆ [true, true]   │
└────────────────┴────────────────┴────────────────┘

In theory this should be a well-defined operation if performed on column with the same width of the pl.Array (for example sum_horizontal could be also defined). In the case of binary array another option is to work with binary string. Any suggestion?

like image 397
S1M0N38 Avatar asked Sep 07 '26 19:09

S1M0N38


1 Answers

explode/implode over a row indicator seems to work. These are copy-free operations btw.

(    df
    .with_columns(
        any=(pl.any_horizontal(pl.col("a").explode(), pl.col("b").explode())
                    .implode()
                    .over(pl.int_range(pl.len()))
            ).cast(pl.Array(shape=2, inner=pl.Boolean))
        )
    )

shape: (2, 3)
┌────────────────┬────────────────┬────────────────┐
│ a              ┆ b              ┆ any            │
│ ---            ┆ ---            ┆ ---            │
│ array[bool, 2] ┆ array[bool, 2] ┆ array[bool, 2] │
╞════════════════╪════════════════╪════════════════╡
│ [false, false] ┆ [true, false]  ┆ [true, false]  │
│ [false, true]  ┆ [true, true]   ┆ [true, true]   │
└────────────────┴────────────────┴────────────────┘
like image 109
Dean MacGregor Avatar answered Sep 10 '26 07:09

Dean MacGregor



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!