Add support for Hibernate about jooq

[qweqq]

Can you provide us with more information on why the idea of supporting HQL was dropped ?

[lukaseder]

Thanks for your message, @qweqq. Heh, issue number #28, that means it originates from the original trac issue tracking system on source forge and was one of the very early ideas.

At the time, the reason why HQL (or rather JPQL) support was dropped was the need to focus on a single feature in this new library, rather than getting distracted by the fuzzy requirements of a multi-language internal DSL. Once HQL is supported, people will also want support for all other kinds of SQL-style dialects, including e.g. Cypher, SOQL, CQL, and many others.

In fact, QueryDSL took this approach, and in my opinion failed at doing so. Just observe how many people on the QueryDSL mailing list get confused by the fact that their JPA query tool doesn't support basic SQL features, such as derived tables. SQL is already a vast field on its own, and jOOQ simply wants to focus on type safe embedded SQL. When querying with SQL, the result is a flat table, not an object graph as in HQL / JPQL, which is rather confusing in Hibernate per se, as HQL / JPQL queires bypass the first / second level caches and query the database directly, so I don't even see their usefulness, in fact.

Now, as far as code generation from Hibernate mappings is concerned, we support that in the meantime, see:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-jpa

We're also integrating more tightly with JPA in every release. For instance (hopefully), jOOQ 3.11 will support fetching entities through JPA's native query API out of the box: #6543

[qweqq]

Thanks for your response!

I was under the impression that HQL/JPQL queries in Hibernate can be cached. According to this blog post (https://vladmihalcea.com/how-does-hibernate-query-cache-work/) if properly configured caching should work.

Our goal is to use JOOQ for complex SQL queries and HQL/JPQL for simple selects with a few joins and maybe a sum() to benefit from the 2nd level caching in Hibernate if we happen to need it.

Currently we are using QueryDSL for both SQL and HQL/JPQL but would like to switch to JOOQ due to the superior SQL support. We will try using JOOQ for SQL and QueryDSL for HQL/JPQL and see how this goes.

[lukaseder]

I was under the impression that HQL/JPQL queries in Hibernate can be cached.

That's not what I meant. What I meant is that HQL/JPQL queries do not query the first/second level caches, they bypass those caches, in case of which I don't see their point.

Our goal is to use JOOQ for complex SQL queries and HQL/JPQL for simple selects with a few joins and maybe a sum() to benefit from the 2nd level caching in Hibernate if we happen to need it.

You can totally use jOOQ to query for entities (select all the entity columns) via Hibernate's native SQL API: https://www.jooq.org/doc/latest/manual/sql-execution/alternative-execution-models/using-jooq-with-jpa/using-jooq-with-jpa-entities

We will try using JOOQ for SQL and QueryDSL for HQL/JPQL and see how this goes.

Sure. Would be very curious to get your feedback on this.

[masc3d]

When querying with SQL, the result is a flat table, not an object graph as in HQL / JPQL, which is rather confusing in Hibernate per se, as HQL / JPQL queires bypass the first / second level caches and query the database directly, so I don't even see their usefulness, in fact.

why would you assume jpql queries bypass caches @lukaseder https://www.eclipse.org/eclipselink/documentation/2.5/jpa/extensions/q_cacheusage.htm

also querying and exposing results are basically two different topics.

I agree the latter (supporting jpa / orm / graph features) doesn't make sense for jooq (at all).
But imho supporting the jpql dialect for querying would.

we actually use jpql / jpa like a flat sql database, but gain abstraction from db referring:

• caching
• entity notifications
• sql dialect

jooq would have been much preferable as we already use it to interface to other databases, we're currently falling back to querydsl for this.

[lukaseder]

why would you assume jpql queries bypass caches

That was an assumption based on my past experience with Hibernate, and on a brief feasibility analysis of such a cache. I may have misunderstood the offered functionality, of course and will be happy to be educated about the status quo. Good to know that EclipseLink offers these flags.

I'm very curious: When your query contains a dozen joins, will a cost based optimiser decide what algorithm to use to query the cached data structure? Will statistics have been gathered and client side indexes be established to ensure that comparable query performance results from running a query on the cache? What if there is a partial cache miss, e.g. 60% of the data is already in the cache and the rest is still in the database? Will a join across systems result, akin to e.g. Oracle database links? What about transactions? Will I see modifications that have been applied in the database but are not reflected in my cache yet?

To me, this EclipseLink feature sounds like it is optimised for trivial queries, and quickly breaks apart for complex ones. Am I perhaps missing something?

also querying and exposing results are basically two different topics.

I agree the latter (supporting jpa / orm / graph features) doesn't make sense for jooq (at all).
But imho supporting the jpql dialect for querying would.

They are different topics, and querying is what SQL is for, as it is really very powerful. A lot of functionality becomes unavailable once JPQL is chosen over SQL. I'm following the JPQL tag on Stack Overflow. There are so many questions on there that would be extremely easy to solve with SQL, and incredibly hard (and non performant) with JPQL.

we actually use jpql / jpa like a flat sql database, but gain abstraction from db referring:

• caching
• entity notifications
• sql dialect

Caching can be achieved in a variety of ways. JPA's cache is one, but have all the options been evaluated? In my experience, entity-based caching is not always the best solution. Often, you want to cache an aggregation result, similar to a client-side materialised view. It can be invalidated, for example, using Oracle AQ.

The SQL equivalent of entity notifications (to my understanding) are triggers, and possibly again a mechanism like Oracle AQ or PostgreSQL notifications.

The SQL dialect is abstracted by jOOQ just the same, but without removing most of the language features for the abstraction purpose.

jooq would have been much preferable as we already use it to interface to other databases, we're currently falling back to querydsl for this.

From your description, I'm not sure if I follow the logic. You prefer jOOQ (SQL) already, yet you also prefer JPQL. Which one is it? :) Why do you prefer jOOQ? Given that jOOQ is all about SQL and not about JPQL, what of jOOQ remains that you prefer?

For jOOQ, supporting alternative query languages and execution models is non trivial. The benefits I had seen so far in using both SQL and JPQL are low. The cost, however, is very high as has been proven by QueryDSL, and Stack Overflow, both in terms of development and support efforts. The number of beginner questions on Stack Overflow and on their mailing list, related to confusion about what particular query should be executed with SQL, and what particular query should be executed with JPQL is tremendous. This is not only excessive effort for maintainers (see QueryDSL's current regrettable maintenance state!), but also a clear sign that the abstraction is not only leaky, but perhaps just simply wrong.

By focusing only on SQL (and I mean ANSI SQL, not some "SQL-esque" language like e.g. Cassandra's query language), jOOQ has been able to maintain a very high quality standard over 11 years now, without being distracted by the many edge cases produced by abstracting over arbitrary query languages.

The only reason jOOQ would ever support more JPA (and we will), is to facilitate migration paths towards more SQL oriented architectures. One such example would be to enable executing SQL queries on the EntityManager through "native queries" and then allowing the EntityManager to materialise entities as a result. Supporting JPQL does not make sense in my opinion.

However, since you do seem to have the use case of combining SQL and JPQL, nothing prevents you from using something like QueryDSL along side with jOOQ. Or you could revert to the Criteria API.

[lukaseder]

... and to provide some additional context: We have a very full roadmap of tons of really cool, completely SQL related features and new products. :) Even if I were more optimistic about the benefits of a SQL/JPQL integration, I think it would not be a priority right now, as I don't see the market going into such a direction.

[masc3d]

To me, this EclipseLink feature sounds like it is optimised for trivial queries, and quickly breaks apart for complex ones. Am I perhaps missing something?

yes. that some applications do "simple" queries for the most part and it perfectly fits the bill.

we use eclipselink on top of h2 as an embedded portable, reactive data store with strong typed query support.

[masc3d]

By focusing only on SQL.. jOOQ has been able to maintain a very high quality standard over 11 years now, without being distracted by the many edge cases produced by abstracting over arbitrary query languages.

Im aware of it and appreciate it.
While I would actually like to see more (non-sql) query language support in general, I do understand your concern that focus may be compromised.

[masc3d]

From your description, I'm not sure if I follow the logic. You prefer jOOQ (SQL) already, yet you also prefer JPQL. Which one is it? :)

same project, different database / use case )

ps. to be more precise we use JOOQ to interface to central database cluster and JPQL for portable database which is used in a distributed node model. for the latter we need caches and notifications / events.

pps. and preference would be clearly to use JOOQ consistently.

[lukaseder]

To me, this EclipseLink feature sounds like it is optimised for trivial queries, and quickly breaks apart for complex ones. Am I perhaps missing something?

yes. that some applications do "simple" queries for the most part and it perfectly fits the bill.

I see. Well, there does seem to be a mismatch in cost / benefit ratio here. The benefit for you could be relatively quickly achieved by supporting the "simple" use cases in JPQL, but there will be users who are more demanding (for maybe entirely different reasons) and would want to see the entirety of JPQL supported as they see little benefit in supporting only the simple things, and then we're back at quite a big maintenance effort, which I would want to avoid.

we use eclipselink on top of h2 as an embedded portable, reactive data store with strong typed query support.

But if your database is embedded, why do you still need a cache? What do your benchmarks / measurements say in this area? Also, H2 supports triggers written in Java, so could those perhaps be used for your notifications?

To me, removing JPA from this equation seems to be a much preferrable solution, now that I know a bit more about your system.

While I would actually like to see more (non-sql) query language support in general, I do understand your concern that focus may be compromised.

What other language(s) would you like to see supported, and why?

same project, different database / use case )

ps. to be more precise we use JOOQ to interface to central database cluster and JPQL for portable database which is used in a distributed node model. for the latter we need caches and notifications / events.

pps. and preference would be clearly to use JOOQ consistently.

Well, I would like that, too, and I will be happy to help. But if all else fails, let me quote Gavin King

Translation to this use case here: "Just because you're using jOOQ doesn't mean you have to use it for everything" 😊

[masc3d]

But if your database is embedded, why do you still need a cache? What do your benchmarks / measurements say in this area? Also, H2 supports triggers written in Java, so could those perhaps be used for your notifications?

yes, h2 does some excellent in memory caching, which makes jpa l2 cache redundant for this particular data store. anyhow we still benefit from l1 caches plus we also leverage query result caches.

and yes Im aware of h2 (java) native triggers.

so what would we gain / lose:

• ++ parity of query apis (which is desirable)
• + a more lightweight setup / process
• -- having to re-invent and maintain own abstraction for notifications
• -- having to re-invent and maintain own l2 cache (which may become relevant when switching to another data store)
• --- losing scalability, as a data store switch in current setup can be estimated in days, will then become weeks to months as above features have to be ported
• --- an otherwise suitable or required data store may not support implementing above features natively at all, thus become unavailable entirely

To me, removing JPA from this equation seems to be a much preferrable solution, now that I know a bit more about your system.

that being said, does not seem preferable to me.

using a well-defined subset of jpa features works really well for us and gives us fancy flexibility being able to switch to anything that supports jdbc at basically zero cost.

I would actually rather implement jooq jpql query support myself, which would be way less work in comparison, way less risky while keeping all the benefits.

Just because you're using jOOQ doesn't mean you have to use it for everything

yes. essential and one of rare guidelines that apply to almost anything )

[lukaseder]

anyhow we still benefit from l1 caches plus we also leverage query result caches.

A query result cache would definitely be useful in jOOQ as well. We'll offer one at some point in the future: #8320

How does your architecture profit from L1 caching?

--- having to re-invent and maintain own l2 cache (which may become relevant when switching to another data store)
--- losing scalability, as a data store switch in current setup can be estimated in days, will then become weeks to months as above features have to be ported
--- an otherwise suitable or required data store may not support implementing above features natively at all, thus become unavailable entirely

This is the same argument 3 times, so I only count it as once ... :) In your architecture, is there a real risk of having to switch from H2 to something else? Why?

(I'm not going to argue here, I just want to better understand the use case)

I would actually rather implement jooq jpql query support myself, which would be way less work in comparison, way less risky while keeping all the benefits.

Well... You argued against implementing your own notification mechanism (should be simple), but in favour of implementing your own language abstraction (not too simple). At some point, this seems like a personal preference of how to spend your time :)

[masc3d]

This is the same argument 3 times, so I only count it as once ... :)

well it's not.

1. is about non-trivial (re-)implementation and maintenance of a feature we already have
2. is about scalablilty and being able to (quickly) respond to new requirements if necessary
3. is about being able to evolve (long term) and avoiding dead ends

you can count it as one if you sum up the minuses ;)

In your architecture, is there a real risk of having to switch from H2 to something else? Why?

yes, there's always risk of tight coupling. because we may need to tackle higher data volume at some point and h2 is really slow in some aspects when volume increases.

Well... You argued against implementing your own notification mechanism (should be simple), but in favour of implementing your own language abstraction (not too simple)

not simple when building from scratch maybe.
but jooq is a strong foundation and this is just a dialect.
and most importantly a one time effort with close to zero risk.

[lukaseder]

not simple when building from scratch maybe. but jooq is a strong foundation and this is just a dialect. and most importantly a one time effort with close to zero risk.

Well, for you, implementing a solution that only has to work for yourself, this is definitely easier than adding such functionality to a product like jOOQ. So you may be right from your perspective.