Comments (4)
We discussed it and we do not want this in Polars, sorry. We consider the with_columns
expression fine.
Note that we already support broadcasting length-wise when multiplying by a series:
>>> df
shape: (3, 4)
┌─────┬─────┬─────┬─────┐
│ a ┆ b ┆ c ┆ d │
│ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 ┆ i64 │
╞═════╪═════╪═════╪═════╡
│ 1 ┆ 4 ┆ 7 ┆ 10 │
│ 2 ┆ 5 ┆ 8 ┆ 11 │
│ 3 ┆ 6 ┆ 9 ┆ 12 │
└─────┴─────┴─────┴─────┘
>>> df * pl.Series([1, 10, 100])
shape: (3, 4)
┌─────┬─────┬─────┬──────┐
│ a ┆ b ┆ c ┆ d │
│ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 ┆ i64 │
╞═════╪═════╪═════╪══════╡
│ 1 ┆ 4 ┆ 7 ┆ 10 │
│ 20 ┆ 50 ┆ 80 ┆ 110 │
│ 300 ┆ 600 ┆ 900 ┆ 1200 │
└─────┴─────┴─────┴──────┘
Adding magic broadcasting column-wise would be very confusing in combination with the already existing above feature.
from polars.
@orlp Thanks for reviewing the request. I see where you're coming from, and it makes sense. What do you think about a named function instead, like how DataFrame
has .sum()
and .sum_horizontal()
?
So the syntax could be something like:
# length-wise broadcast
df.multiply(pl.Series([1,2,3,4]))
# column-wise broadcast
df.multiply_horizontal(pl.Series([1,2,3,4))
I think that is more clear than
df.with_columns([pl.col(col) * multiplier for col, multiplier in zip(df.columns, [1,2,3,4])])
The reason I hope you would reconsider is that broadcasts along rows & columns is very common in both Numpy and Pandas. Users will be familiar with the concept via the axis
parameter in those libraries. In Polars it looks like that parameter was removed, but the utility of broadcasting in multiple axes remains incredibly valuable.
Of course, column-wise broadcast could be trivially done by taking the transpose of the polars DataFrame and using your supplied syntax, but the documentation for the transpose method discourages its use:
This is a very expensive operation. Perhaps you can do it differently.
Given the current Polars API, my workaround is to convert to Pandas to perform that one line operation for readability purposes, before converting it back to Polars. I think this is a bit goofy but I probably won't be the only one doing this.
In any case, I hope there can be a more readable and concise solution than what we currently have.
from polars.
nth
could perhaps tidy up the current with_columns
approach a bit:
df.with_columns(pl.nth(n) * m for n, m in enumerate(multipliers))
from polars.
nth
could perhaps tidy up the currentwith_columns
approach a bit:df.with_columns(pl.nth(n) * m for n, m in enumerate(multipliers))
Thanks that is better than before -- this could be good in the migration guide for Polars.
Having thought about it a bit more, if the Polars developers don't like sum_horizontal
as it creates a precedence to have sister functions suffixed with _horizontal
for other library functions, then I can see why the multiply_horizontal
idea is disfavored.
For now I'll stick with Pandas' syntax, as a personal preference.
from polars.
Related Issues (20)
- `Series.search_sorted` gives wrong answer when using expression arguments HOT 4
- Panic occurring when using streaming and limit with Parquet
- `Expr.replace` and `Expr.replace_strict` set "NO_DEFAULT" as a value HOT 1
- Parquet files cannot be read from pre-signed S3 URLs due to S3 forbidding HTTP HEAD HOT 8
- Wrong result when filtering categorical using `.is_in` in `scan_parquet` HOT 2
- `.over()` fails with `.get`. HOT 6
- Don't disable coalesce when joining on expressions HOT 2
- entered unreachable code when using collect_schema and concat HOT 1
- Use of write_ipc with file=None
- Examples of using `scan_csv` with cloud URIs HOT 2
- Consistency in check_exact between assert_frame_equal and polars.DataFrame.equals
- `compat_level` documentation is unclear
- Request to return LazyFrame for pl.from_arrow and more HOT 2
- rolling_corr giving inconsistent results
- `test_read_database_cx_credentials` expected exception does not bifurcate over correct Python version HOT 1
- In `scan_parquet()`, `include_file_paths` returns twice the same column
- In `bin.size()`, the `unit` parameter is not described HOT 1
- `.implode().list` method is not valid before `.over`. HOT 3
- Upgrading to 0.42 breaks compilation HOT 4
- Expr `.implode().get().over()` has strange results. HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polars.