Support for hidden dependencies about maven-mvnd

[ppalaga]

Let me try if the smart builder is considering the plugin dependencies when building the dependency tree. Adding -deployment modules as dependencies of some plugin could be a viable workaround.

[ppalaga]

I thought using the code in https://github.com/gnodet/mvnd/pull/13 and building with -T1 would solve the problem, but no, it looks like the ordering of the modules is non-deterministic and build succeeds only sometimes. Interesting.

[ppalaga]

-T1 -bsinglethreaded seems to work consistently as a workaround.

[gnodet]

Yes, that's one of the problem with a parallel build: all dependencies must be explicitely defined, as we can't rely on the order in which modules are defined.

[ppalaga]

Yes, but Quarkus extensions are a bit special in this respect. The deployment module is not declared as a dependency by design. It does not suit any of the traditional Maven scopes. It is actually only pulled by the Quarkus plugin to be run at built time. So declaring it as a dependency of the Quarkus plugin seems be closest to what it really is (although the plugin does not require that). I will try whether the smart builder will honor the declaration and schedule the deployment module before the itest.

But anyway shouldn't mvnd offer an official way to adjust the build order? I can imagine many projects will have similar problems. E.g. a system property? Something like

<groupId>my-org</groupId>
<artifactId>my-itest</artifactId>
...
<properties>
  <mvnd.schedule-after>my-org:ext1-deployment,my-org:ext2-deployment</mvnd.schedule-after>
</properties>

[ppalaga]

mvnd.schedule-after is stupid. mvnd.schedule.after would be more consistent.

[ppalaga]

Adding the deployment deps as plugin deps seems to solve the scheduling problem, but the effective result seems to be a corrupted local repo. Thinking so because the stock mnv is throwing the same errors in java.io that go away (with the stock maven) when I delete the broken jars from the local repo.

Steps to reproduce should be

1. Checkout this branch https://github.com/ppalaga/camel-quarkus/commits/191003-hidden-deps
2. Remove all quarkus 2.23.2 artifacts from your local repo, e.g. using find . -type d -name 0.23.2 -exec rm -Rf {} \;
3. mvnd clean install -DskipTests

[ppalaga]

The failures in the previous comment are caused by the fact that the Quarkus plugin does not like to have the deployment modules in its class path. So I tried to add the deployment modules as deps of another plugin see here https://github.com/ppalaga/camel-quarkus/commits/191003-hidden-deps.1 and it works! mvnd clean install -DskipTests took ~26 sec instead of ~57 sec with the stock Maven!

[ppalaga]

So adding the hidden deps to some plugin is a valid workaround, but the question is still open whether mvnd should offer an official way to reach the goal without having to add a plugin.

[gnodet]

So adding the hidden deps to some plugin is a valid workaround, but the question is still open whether mvnd should offer an official way to reach the goal without having to add a plugin.

I agree we should provide a way to add dependencies for ordering purposes. Using properties definitely looks like a good way. However, I'd rather move such information in the parent pom rather than dispatched in the various modules. This would also enable to use wildcards to define those additional rules. This would also benefit camel which also has similar problems.

[ppalaga]

I'd rather move such information in the parent pom

I wonder what would it look like?

Would those be honored also when not building from the root dir?

Having it in the dependent module seems to have the following advantages:

• No need to write the dependent module, because it is implicitly clear - it is "this" module
• More robust when renaming/removing the dependent module

[gnodet]

I've added support for top level rules for now. The syntax is the following:

<rules> ::= rule [ <rule-sep> <rule> ]+
<rule-sep> ::= '\n' | ';'
<rule> ::= <projects> <sep> <projects>
<sep> ::= ' before ' | ' -> '
<projects> ::= <project> [ <project-sep>  <project> ]+
<project-sep> ::= ',' | 'and'
<project> ::= [ <groupId> ':' ]? <artifactId>

The groupId and artifactId supports the * wildcard matcher. If the groupId part is omitted, * is implied.

The rules have to be defined as the mvnd.builder.rules maven property.

[gnodet]

A simple top level rule is

org.apache.camel.quarkus:*-deployment before org.apache.camel.quarkus.tests:*

[gnodet]

We could also add support for non top-level rules where the right part of the rule would be implied to the current groupId:artifactId.

[gnodet]

Just added support for local rules defined using mvnd.builder.rule where the right part is implied.