Ability to set session properties about dbt-trino

[findinpath]

Thank you @Btwitzke for raising this issue.

Do you have any suggestion on where to specify the session properties for Trino within dbt ?

What do you think about specifying them, in the same fashion as the http_headers within the trino dbt profile?

dbt-labs/dbt-presto#68

[Btwitzke]

Thanks for the quick response @findinpath !

I think that there are 2 ways of looking at this:

1. As you mentioned, since this is going into the actual connection then it could be specified in the trino dbt profile along with the rest of the connection details. This would be a good place to start for this.

2. To expand functionality in another step, it is sometimes desired to specify different session variables for different models, rather than a wholesale connection variable in the profile. An example is raising the amount of task writers when knowing that a model produces larger data files and not wanting so many writers when a model has a small amount of data. In that sense, the optimal way would be the ability to specify session variables in the same type of 'pre-hook' syntax within a model or in the dbt_project.yml.

[findinpath]

@Btwitzke the 2nd point you've raised points rather to a one off "hint" for Trino that you want to use for a specific query.

Do you have a recipe for such a scenario when interacting directly with Trino?

dbt offers pre-hook and post-hook
https://docs.getdbt.com/reference/resource-configs/pre-hook-post-hook

which can be used to SET SESSION / RESET SESSION

I still need to test this, but i assume that this feature request for one off queries doesn't require any implementation.
Global session properties would probably make sense to be implemented via dbt/profiles.yml config file.

[Btwitzke]

@findinpath Yes, a hint it a way to put it. Setting the session variables within a pre-hook currently does not actually effect the running queries. It appears that upon a new transaction, a new connection is made which resets the session properties of course. I tested running 'set session' queries in a pre-hook, and these do NOT persist into the corresponding model query. I think that is because a pre-hook is actually run as a separate query before the model.

If there was the ability to set session properties in the profiles.yml that would also work for this case, and most likely the simplest solution since those variables essentially already persist into the connections made.

[findinpath]

@Btwitzke I had a look on the dbt logs when testing the changes from @hovaesco and it seems that after each query, a new Trino connection is being opened:

Opening a new connection, currently in state closed

This generate the loss of the settings set on pre-hook.

For now, we can use the session properties set in the profiles.yml dbt file.

Please do create a new issue if you have an idea on how to make use of session properties that are to be applied only on a specific query.