Error using readRecords to read a database query. about nodejs-polars

[jospueyo]

Debugging a bit I found that the problem is with the null values. However, it is expected to have null values in a database. So, I don't know if this should be tagged as a bug.

[universalmind303]

The inference used to detect the schemas for readRecords is a bit behind the upstream polars. Updating that to use the latest code from polars should likely fix most inference related issues.

the nodejs readRecords should behave identically to the python from_dicts. So if you are able to create using python from_dicts, then it is definitely a bug in nodejs-polars. Otherwise, i'd suggest opening up an issue with polars.

[jospueyo]

I tried in both languages and in Python works properly:

Python:

import polars as pl

test = [
    {"a": 1, "b": "a", "c": "x"},
    {"a": 2, "b": None, "c": "y"},
]

test_df = pl.from_dicts(test)

print(test_df)

# shape: (2, 3)
#┌─────┬──────┬─────┐
#│ a   ┆ b    ┆ c   │
#│ --- ┆ ---  ┆ --- │
#│ i64 ┆ str  ┆ str │
#╞═════╪══════╪═════╡
#│ 1   ┆ a    ┆ x   │
#│ 2   ┆ null ┆ y   │
#└─────┴──────┴─────┘

Nodejs:

const pl = require("nodejs-polars")

let test = [
    {a: 1, b: "a", c:"x"},
    {a: 2, b: null, c: "y"}
]

let testDF = pl.readRecords(test)

console.log(testDF)

// thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', src\dataframe.rs:1527:29
// note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

However, it works if the null values are dropped:

const pl = require("nodejs-polars")

let test = [
    {a: 1, b: "a", c:"x"},
    {a: 2, c: "y"}
]

let testDF = pl.readRecords(test)

console.log(testDF)

// shape: (2, 3)
// ┌─────┬──────┬─────┐
// │ a   ┆ b    ┆ c   │
// │ --- ┆ ---  ┆ --- │
// │ f64 ┆ str  ┆ str │
// ╞═════╪══════╪═════╡
// │ 1.0 ┆ a    ┆ x   │
// ├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌┤
// │ 2.0 ┆ null ┆ y   │
// └─────┴──────┴─────┘

[Bidek56]

This issue is fixed in 0.8.0.
Please try it, if it works, please close the ticket. Thx