Query Errors if pg_hint_plan.enable_hint_table and enable_hint is enabled but hints table does not exist about pg_hint_plan

[samimseih]

I took another look. If the user creates the pg_hint_plan extension, this will not be an issue as the hints table is created as part of the extension.
It also seems there will be difficulty in actually handling this error and proceeding with the query as the user must rollback the transaction.

At minimum, should this scenario be highlighted in docs [1]?

Thoughts?

[1] https://github.com/ossc-db/pg_hint_plan/blob/master/docs/hint_table.md#the-hint-table

[michaelpq]

One thing that we could do is a check callback for the GUC to ease the error message when using a SET. Like for check_default_table_access_method, it cannot be perfect: it is not possible to do a catalog lookup to see if the hint table exists or if the extension is installed if not connected to a database and if not within a transaction.

[samimseih]

One thing that we could do is a check callback for the GUC to ease the error message when using a SET. Like for check_default_table_access_method, it cannot be perfect:

To be sure, you are thinking to check if the pg_hint_plan extension has been created in the guc check hook?

[michaelpq]

To be sure, you are thinking to check if the pg_hint_plan extension has been created in the guc check hook?

Or check that the table exists. Both require a catalog lookup and a transaction state, so that could run under the same constraints. LImited because of these constraints, still perhaps useful for some.

[rjuju]

such a check would indeed be very limited, and totally useless in some scenarios that I would expect to be common (like using session_preload_libraries). it also wouldn't cover e.g. persistent connections and a later drop extension.

it would probably be better to add a check in get_hints_from_table() that the table exist (like trying to get its oid or something) and raise a clear error message with a hint.

[michaelpq]

Hmm. Is an error message the best choice though? If pg_hint_plan.enable_hint_table is enabled in postgresql.conf, then one would get an error for basically all queries run in the backend. I have been annoyed by that a bit myself. So how about lowering this log's level? WARNING would noisy and bloat the logs, though.

[rjuju]

On the other hand, if you rely on the extension being installed to not have your production server crash and burn due to incorrect plans, downgrading the message to WARNING would decrease the chances to notice the problem immediately after a maintenance operation.

Ideally we should document that enabling it globally is likely to be a terrible idea and should be done at the database level using ALTER DATABASE SET.

[samimseih]

On the other hand, if you rely on the extension being installed to not have your production server crash and burn due to incorrect plans, downgrading the message to WARNING would decrease the chances to notice the problem immediately after a maintenance operation.

That's a good point. It's better to error out rather than let query execution go sideways leaving the user confused as to why. Adding more documentation are this behavior is necessary at minimum.

[michaelpq]

Ideally we should document that enabling it globally is likely to be a terrible idea and should be done at the database level using ALTER DATABASE SET.

Yeah, agreed that this looks like a good alternative based on the fact that CREATE EXTENSION needs to happen for each database.

[samimseih]

I've also wondered if the hints table can be turned into a dynamic-sized hash table; which may be a better way to go long term particularly if we remove the normalized_text and start using queryId. We will have to deal with persisting the data across reboots and crashes.

[rjuju]

We could do that, but there will still be the fundamental requirement that query identifiers are only guaranteed to be meaningful on a per-database level (unless you have a jumbling extension that only works with object names rather than oid, but we can't assume that by default), so you will need to add the database oid in the hash key and declare them on a per-database basis anyway. Given those requirements I'm not sure that the extra infrastructure to handle it will be really worth it.

[samimseih]

It may still be worth considering.

There could probably be performance benefits with going down the hash table path, such as the data always being in memory ( a heavily used hints table will probably always be in memory, so that's probably not a major gain), but there is also the additional locking required for a normal table and the extra sql to fetch the hint.

[michaelpq]

Given those requirements I'm not sure that the extra infrastructure to handle it will be really worth it.

I guess that you are also referring here to the part where the data would need to persist across restarts and fill in the hash table when starting without crash recovery, right?

[rjuju]

I guess that you are also referring here to the part where the data would need to persist across restarts and fill in the hash table when starting without crash recovery, right?

Exactly. And unless we want to do the configuration using some SQL wrappers on top of C functions and declare them manually on each replica, the source of truth will probably stay in some local table, so this would only be a cache so it will also need to handle eviction and refreshing based on the source table, handle the MVCC aspect and so on.

Having such a cache will likely improve performance, as @samimseih pointed, but I don't think it's really trivial to do.

[michaelpq]

Having such a cache will likely improve performance, as @samimseih pointed, but I don't think it's really trivial to do.

I'm biased on the risk vs benefit risk on this one and there are a few other things I'd like to get done around first, so I don't really plan to work on that.