Broadcast operations similar to Pandas / Numpy about polars

[orlp]

We discussed it and we do not want this in Polars, sorry. We consider the with_columns expression fine.

Note that we already support broadcasting length-wise when multiplying by a series:

>>> df
shape: (3, 4)
┌─────┬─────┬─────┬─────┐
│ a   ┆ b   ┆ c   ┆ d   │
│ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 ┆ i64 │
╞═════╪═════╪═════╪═════╡
│ 1   ┆ 4   ┆ 7   ┆ 10  │
│ 2   ┆ 5   ┆ 8   ┆ 11  │
│ 3   ┆ 6   ┆ 9   ┆ 12  │
└─────┴─────┴─────┴─────┘
>>> df * pl.Series([1, 10, 100])
shape: (3, 4)
┌─────┬─────┬─────┬──────┐
│ a   ┆ b   ┆ c   ┆ d    │
│ --- ┆ --- ┆ --- ┆ ---  │
│ i64 ┆ i64 ┆ i64 ┆ i64  │
╞═════╪═════╪═════╪══════╡
│ 1   ┆ 4   ┆ 7   ┆ 10   │
│ 20  ┆ 50  ┆ 80  ┆ 110  │
│ 300 ┆ 600 ┆ 900 ┆ 1200 │
└─────┴─────┴─────┴──────┘

Adding magic broadcasting column-wise would be very confusing in combination with the already existing above feature.

[u3Izx9ql7vW4]

@orlp Thanks for reviewing the request. I see where you're coming from, and it makes sense. What do you think about a named function instead, like how DataFrame has .sum() and .sum_horizontal()?

So the syntax could be something like:

# length-wise broadcast
df.multiply(pl.Series([1,2,3,4]))

# column-wise broadcast
df.multiply_horizontal(pl.Series([1,2,3,4))

I think that is more clear than

df.with_columns([pl.col(col) * multiplier for col, multiplier in zip(df.columns, [1,2,3,4])])

The reason I hope you would reconsider is that broadcasts along rows & columns is very common in both Numpy and Pandas. Users will be familiar with the concept via the axis parameter in those libraries. In Polars it looks like that parameter was removed, but the utility of broadcasting in multiple axes remains incredibly valuable.

Of course, column-wise broadcast could be trivially done by taking the transpose of the polars DataFrame and using your supplied syntax, but the documentation for the transpose method discourages its use:

This is a very expensive operation. Perhaps you can do it differently.

Given the current Polars API, my workaround is to convert to Pandas to perform that one line operation for readability purposes, before converting it back to Polars. I think this is a bit goofy but I probably won't be the only one doing this.

In any case, I hope there can be a more readable and concise solution than what we currently have.

[cmdlineluser]

nth could perhaps tidy up the current with_columns approach a bit:

df.with_columns(pl.nth(n) * m for n, m in enumerate(multipliers))

[u3Izx9ql7vW4]

nth could perhaps tidy up the current with_columns approach a bit:

df.with_columns(pl.nth(n) * m for n, m in enumerate(multipliers))

Thanks that is better than before -- this could be good in the migration guide for Polars.

Having thought about it a bit more, if the Polars developers don't like sum_horizontal as it creates a precedence to have sister functions suffixed with _horizontal for other library functions, then I can see why the multiply_horizontal idea is disfavored.

For now I'll stick with Pandas' syntax, as a personal preference.