jmcnamara / polars_excel_writer

A Polars extension to serialize dataframes to Excel xlsx files

polars_excel_writer

The polars_excel_writer crate is a library for serializing Polars dataframes to Excel Xlsx files.

It provides two interfaces for writing a dataframe to an Excel Xlsx file:

• ExcelWriter a simple Excel serializer that implements the Polars SerWriter trait to write a dataframe to an Excel Xlsx file.

• PolarsXlsxWriter a more configurable Excel serializer that more closely resembles the interface options provided by the Polars write_excel() dataframe method.

ExcelWriter uses PolarsXlsxWriter to do the Excel serialization which in turn uses the rust_xlsxwriter crate.

Example

An example of writing a Polar Rust dataframe to an Excel file using the ExcelWriter interface.

use chrono::prelude::*;
use polars::prelude::*;

fn main() {
    // Create a sample dataframe for the example.
    let mut df: DataFrame = df!(
        "String" => &["North", "South", "East", "West"],
        "Integer" => &[1, 2, 3, 4],
        "Float" => &[4.0, 5.0, 6.0, 7.0],
        "Time" => &[
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            ],
        "Date" => &[
            NaiveDate::from_ymd_opt(2022, 1, 1).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 2).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 3).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 4).unwrap(),
            ],
        "Datetime" => &[
            NaiveDate::from_ymd_opt(2022, 1, 1).unwrap().and_hms_opt(1, 0, 0).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 2).unwrap().and_hms_opt(2, 0, 0).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 3).unwrap().and_hms_opt(3, 0, 0).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 4).unwrap().and_hms_opt(4, 0, 0).unwrap(),
        ],
    )
    .unwrap();

    example1(&mut df).unwrap();
    example2(&df).unwrap();
}

// The ExcelWriter interface.
use polars_excel_writer::ExcelWriter;

fn example1(df: &mut DataFrame) -> PolarsResult<()> {
    let mut file = std::fs::File::create("dataframe.xlsx").unwrap();

    ExcelWriter::new(&mut file).finish(df)
}

// The PolarsXlsxWriter interface. For this simple case it is
// similar to the ExcelWriter interface but it has additional
// options to support more complex use cases.
use polars_excel_writer::PolarsXlsxWriter;

fn example2(df: &DataFrame) -> PolarsResult<()> {
    let mut xlsx_writer = PolarsXlsxWriter::new();

    xlsx_writer.write_dataframe(df)?;
    xlsx_writer.save("dataframe2.xlsx")?;

    Ok(())
}

Second output file (same as the first):

See also

• Changelog: Recent additions and fixes.
• Performance: Performance comparison with Python based methods.

polars_excel_writer's Issues

Bug: Version 0.3.0 on Crates.io - `the trait From<XlsxError> is not implemented for polars::prelude::PolarsError`

Current behavior

Looks like cargo check / build is returning a trait related error when tying it use version 0.3.0 and 0.2.0 on Crates.io.
https://crates.io/crates/polars_excel_writer/versions

Error:

$ cargo check
    Checking polars_excel_writer v0.3.0
error[E0277]: `?` couldn't convert the error to `polars::prelude::PolarsError`
   --> /home/adelg003/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars_excel_writer-0.3.0/src/xlsx_writer.rs:468:33
    |
468 |         self.workbook.save(path)?;
    |                                 ^ the trait `From<XlsxError>` is not implemented for `polars::prelude::PolarsError`
    |
    = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
    = help: the following other types implement trait `From<T>`:
              <polars::prelude::PolarsError as From<regex::error::Error>>
              <polars::prelude::PolarsError as From<std::io::Error>>
              <polars::prelude::PolarsError as From<polars::error::ArrowError>>
    = note: required for `Result<(), polars::prelude::PolarsError>` to implement `FromResidual<Result<Infallible, XlsxError>>`

Expected behavior

When installing polars_excel_writer from crates.io, the versions available should be free of the From<XlsxError> is not implemented for polars::prelude::PolarsError error.

Sample code to reproduce

Use the following version in the Cargo.toml:

polars_excel_writer = "0.3.0"

or

polars_excel_writer = "0.2.0"

Environment

- polars_excel_writer version: 0.3.0 and 0.2.0
- rustc version: rustc 1.74.0 (79e9716c9 2023-11-13)
- Excel version: N/A
- OS: Tested on Manjaro Linux and Mac (Intel version)

Any other information

Ironically, it looks like the current commit (31f09fb) in the main branch does not have this issue.

Version with error from Crates.io:

polars_excel_writer = "0.3.0"

and

polars_excel_writer = "0.2.0"

(0.1.0 does not have this error)

Working version from github:

polars_excel_writer = { git = 'https://github.com/jmcnamara/polars_excel_writer' }

May make sense to update the version present in crates.io with what is current in the main branch and yank versions 0.2.0 and 0.3.0.

If more information is needed, please let me know.

feature request: <map to supported integers>

Feature Request

Good morning!!!

I Try to use

fn write_xlsx(df: &DataFrame) -> PolarsResult<()> {
    let mut xlsx_writer = PolarsXlsxWriter::new();

    xlsx_writer.write_dataframe(df)?;
    xlsx_writer.save("df_result.xlsx")?;

    Ok(())
}

And I get this Error:

Error: ComputeError(ErrString("Polars AnyValue data type 'i64' is not supported by Excel"))

Is it possible to map 'i64' and all unsupported integers to 'u32' supported by Excel?

Roadmap

The polars_excel_writer crate is a library for serializing Polars dataframes
to Excel Xlsx files.

It provides two interfaces for writing a dataframe to an Excel Xlsx file:

• ExcelWriter a simple Excel serializer that implements the Polars
  SerWriter trait to write a dataframe to an Excel Xlsx file.

• PolarsXlsxWriter a more configurable Excel serializer that more closely
  resembles the interface options provided by the Polars write_excel()
  dataframe method.

ExcelWriter uses PolarsXlsxWriter to do the Excel serialization which in
turn uses the rust_xlsxwriter crate.

Example

An example of writing a Polar Rust dataframe to an Excel file using the
ExcelWriter interface.

use chrono::prelude::*;
use polars::prelude::*;

fn main() {
    // Create a sample dataframe for the example.
    let mut df: DataFrame = df!(
        "String" => &["North", "South", "East", "West"],
        "Integer" => &[1, 2, 3, 4],
        "Float" => &[4.0, 5.0, 6.0, 7.0],
        "Time" => &[
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            NaiveTime::from_hms_milli_opt(2, 59, 3, 456).unwrap(),
            ],
        "Date" => &[
            NaiveDate::from_ymd_opt(2022, 1, 1).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 2).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 3).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 4).unwrap(),
            ],
        "Datetime" => &[
            NaiveDate::from_ymd_opt(2022, 1, 1).unwrap().and_hms_opt(1, 0, 0).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 2).unwrap().and_hms_opt(2, 0, 0).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 3).unwrap().and_hms_opt(3, 0, 0).unwrap(),
            NaiveDate::from_ymd_opt(2022, 1, 4).unwrap().and_hms_opt(4, 0, 0).unwrap(),
        ],
    )
    .unwrap();

    example1(&mut df).unwrap();
    example2(&df).unwrap();
}

// The ExcelWriter interface.
use polars_excel_writer::ExcelWriter;

fn example1(df: &mut DataFrame) -> PolarsResult<()> {
    let mut file = std::fs::File::create("dataframe.xlsx").unwrap();

    ExcelWriter::new(&mut file).finish(df)
}

// The PolarsXlsxWriter interface. For this simple case it is similar to the
// ExcelWriter interface but it has additional options to support more complex
// use cases.
use polars_excel_writer::PolarsXlsxWriter;

fn example2(df: &DataFrame) -> PolarsResult<()> {
    let mut writer = PolarsXlsxWriter::new();

    writer.write_dataframe(df)?;
    writer.write_excel("dataframe2.xlsx")?;

    Ok(())
}

Second output file (same as the first):

Performance

The table below shows the performance of writing a dataframe using
Python Polars, Python Pandas and PolarsXlsxWriter.

• Performance data:

  Test Case	Time (s)	Relative (%)
  Polars	6.49	100%
  Pandas	10.92	168%
  polars_excel_writer	1.22	19%
  polars_excel_writer + zlib	1.08	17%

The tested configurations were:

• Polars: The dataframe was created in Python Polars and written using the
  write_excel() function. See perf_test.py.
• Pandas: The dataframe was created in Polars but converted to Pandas and
  then written via the Pandas to_excel() function. See also
  perf_test.py.
• polars_excel_writer: The dataframe was created in Rust Polars and
  written using the PolarsXlsxWriter interface. See perf_test.rs.
• polars_excel_writer + zlib: Same as the previous test case but uses
  the zlib feature flag to enable the C zlib library in conjunction with
  the backend ZipWriter.

Note: The performance was tested for the dataframe writing code only.
The code used to create the dataframes was omitted from the test results.