Qryn crash under load - ERR_STREAM_PREMATURE_CLOSE about qryn

[jpsfs]

Thank you for the follow-up @lmangani !
Only normal ClickHouse logs, the query itself doesn't seem to fail on ClickHouse and if I try to execute it manually it succeeds quite fast.

I forgot to mention that this was tested in the latest version (released today) as well as in the previous two versions.

If the database is smaller (less data) the query succeeds on qryn as well.

Best,
José

[lmangani]

Thanks for the report @jpsfs
Do you see any logs or errors from ClickHouse as this query fails?

[akvlad]

@jpsfs do you have an error message: "timeout" in Grafana when you request histogram_quantile(0.50, ....) ?

[jpsfs]

Yes, as qryn crashes and the request in grafana gets a timeout. Best,

…

On Fri, Jun 14, 2024, 11:14 akvlad ***@***.***> wrote: @jpsfs <https://github.com/jpsfs> do you have an error message: "timeout" in Grafana when you request histogram_quantile(0.50, ....) ? — Reply to this email directly, view it on GitHub <#516 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAH6GOOGWMVIJITDP7MGT5TZHK67VAVCNFSM6AAAAABJJF65TCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRXG4YDQMZZHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[lmangani]

@jpsfs could you please retest using the latest release and provide any feedback?

EXPERIMENTAL_PROMQL_OPTIMIZE==1

[akvlad]

Hello @jpsfs . 3.2.24 version is released.

• sum and rate functions were optimized using clickhouse functions.

Please set the env var EXPERIMENTAL_PROMQL_OPTIMIZE=1 before usage.
Please share the user experience of using sum and rate functions (like in your histogram_quantile.... request) so we can decide if the further optimizations are worth to be done.