Performance issue occurs when dataset has a parameter mapped to int8 type on PostgreSql about birt

[speckyspooky]

I understand the mentioned topic with the time differences and the SQL executions.
But by default of course it is not a typical BIRT issue or a BIRT enhancement because the handling will be done based on the JDC-prepared statement and your used JDBC driver.

The only thing what you can do for example to qualify your parameter at the dataset as "Integer"
(in the most cases it will be set to String insetad of the mentioned data type).
Then the parameter will be set as prepared-parameter with datatype "Integer" to the prepared-statement.

But here starts the Java-JDBC-level. I don't know if the JDBC driver will handle "Integer" according to your used type "int8".

[hvbtup]

Using FLOAT is definitely not a good idea here.

But there should be a way to set jdbc parameter type as bigint or long (8 byte). So driver may map the right type to execution.

From a user's perspective, you are certainly right, but as Thomas wrote:

The changed to support "long" / "bigint" would be no small change

So don't expect this enhancement to actually happen unless you can provide a PR to implement this (including tests).

What I do to perform type conversion in BIRT queries is to use this pattern:

-- This is Oracle syntax, but you can easily adapt this for other SQL dialects
with params as
(
  select :p1 as foo,
         cast(:p2 as whatever_type) as bar,
         to_date(:p3, 'DD.MM.YYYY') as some_date
  from dual
)
select ...
from params
join x on (x.foo = params.foo)
...

I mean, I use each bind variable exactly once, and when I need to convert the type, I also have to perform the conversion only once.

I'm not an expert on PostgreSQL, but I guess what happens is:

The DB sees a conversion between an int8 column and a numeric bind variable.
Now, implicit type coercion says that the numeric type is "broader" than int8 and thus the int8 side has to be converted to numeric. Unfortunately, this is the side with millions of values. These have to be converted for all rows, and (that's a wild guess:) maybe not even an index on that column can be used for the query due to the (conversion) function call.

So, since an enhancement to BIRT seems unlikely, your solution with the explicit cast or the very similar approach I mentioned, is probably the best way to go.

As an alternative, you could also modify the SQL itself in the beforeOpen event to write the values literally into the statement. But you have to be careful then, because that might open the door for SQL injection attacks. OTOH this way the query optimizer can see whether you only need the data for yesterday, or for the last 10 years, which makes a huge difference for the optimal execution plan, if there is an index for the column (full table scan or access by index).

[speckyspooky]

Ok, in your case the integer is a long-integer.
Have you tried to use decimal / float to get the value exchanged?

The changed to support "long" / "bigint" would be no small change because it has eimpact on parameter side and dataset side.
Therefore if "decimal" or "float" would be work to then we would have currently a solution of it.

[merks]

@mgulden FYI, setting up a BIRT development environment for contribution is automated:

https://github.com/eclipse-birt/birt?tab=readme-ov-file#create-a-birt-development-environment

[mgulden]

To get rid of this issue, i think maping between jdbc types and postgresql types is crutiual. As you mention this is drivers behaviour. But there should be a way to set jdbc parameter type as bigint or long (8 byte). So driver may map the right type to execution.

I haven't use integer dataset parameter type because i thought 4 byte. Birt gives error if i select integer.

If there is a way to add BIGINT option to dataset parameter and send this parameter's value to jdbc as long java type the problem might be fixed. Here i some findings about type mappings.

Ref: https://www.instaclustr.com/blog/postgresql-data-types-mappings-to-sql-jdbc-and-java-data-types/

Best regards

[mgulden]

It works with DECIMAL or FLOAT dataset parameter type, but it causes significant performance issue which is hard to understand and fix. Thats why i created the issue.

[speckyspooky]

Yes, I confirm the "best practice" of @hvbtup.
I use exactly the same for my PL/SQL-statements and fro T/SQL I use a modified version of it but with the same way.

-- T-SQL:
DECLARE @param01 BIGINT = TRY_CAST(? AS BIGINT);
DECLARE @param02 DECIMAL(10,2) = TRY_CAST(? AS DECIMAL(10, 2));

SELECT ...
FROM myTable
WHERE
  ID = @param01
  AND amount = @param02
;

[mgulden]

So don't expect this enhancement to actually happen unless you can provide a PR to implement this (including tests).

I wish i could send PR but i have no development experience on Birt project.

What I do to perform type conversion in BIRT queries is to use this pattern:

-- This is Oracle syntax, but you can easily adapt this for other SQL dialects
with params as
(
  select :p1 as foo,
         cast(:p2 as whatever_type) as bar,
         to_date(:p3, 'DD.MM.YYYY') as some_date
  from dual
)
select ...
from params
join x on (x.foo = params.foo)
...

Using parameters as join source is a new approach for me, thank you.

[mgulden]

DECLARE @Param01 BIGINT = TRY_CAST(? AS BIGINT);
DECLARE @param02 DECIMAL(10,2) = TRY_CAST(? AS DECIMAL(10, 2));

Postgresql does allow parameter definitions only in functions.

For PostgreSql, the solution in my first post is working.

I wish i see this enhancement for the future roadmap.

I may involve the development and send PR if project admin assign me a budy to help setting up development environment, and describe code organization :)

[hvbtup]

The syntax :p1 is just an Oracle specific syntax for named bind variables. This is equivalent to ? in JDBC.

In fact, at least for BIRT, this is limited in comparison to other clients / programming languages. In other clients/languages, one can use :p1 several times in the statement, while still supplying a value for it only once. This does not work in BIRT. Basically, in BIRT, one can assume that each :variable (except in comments) is literally replaced with a ?.
Working around this limitation is another good reason for the with params (...) idiom.

BTW the from dual clause is not needed in some other DBs (dual is a table with exactly one row)

[speckyspooky]

Each new contributer is welcome to the BIRT project.

@mgulden
If you start a change for that please think that the new data type "Long" should be available not only on parameter side and on JDBC-parameter-side.
Because there are many other objects which use the available/supported data types like integer, decimal and so on.

So different topics should be adjusted of it like:

• scalar parameters
• cascading parameter
• JDBC-parameters
• dataset result column data types
• data-element data tyes (based elements of tables)
• data cubes
• chart data types
• etc.

The current problem can be solved through explicit parameter-cast on SQL-side so I will close this ticket.