Stop using `tool` about pipeline-library

[jglick]

What is the issues with using tool?

It is not tool itself per se, it is the downloads from every build to some Internet location controlled by someone else—a URL that could break at any time, or suddenly impose rate limiting to the ci.jenkins.io IP address, etc. And it is pretty silly when we have the required tool already available to the agent—we just need to point to it.

If we adjust the path, is it ok to define the PATH/JAVA_HOME variables at agent level

Well that would be the option to adjust images. I am guessing it would be simpler to just adjust buildPlugin.groovy etc. to select the desired tool path from a single image.

[jglick]

not sure if needed for container images as they will only have one java version installed.

Right, we only need to change how VM agents work:

pipeline-library/vars/buildPlugin.groovy

Line 44 in 83d5819

boolean addToolEnv = !useContainerAgent

[timja]

We should pack the JDK & Maven into VMs, not just container images, or at a minimum offer our own (e.g., Artifactory) download locations rather than using public URLs.

We already do that, should be fine to just remove them.

[jglick]

We already do that

And are they in $PATH / %Path%? If so it would suffice to simply delete tool steps and associated code.

[timja]

hmm, https://github.com/jenkins-infra/packer-images/blob/main/scripts/ubuntu-20-provision.sh#L121-L139

https://github.com/jenkins-infra/packer-images/blob/main/scripts/windows-2019-provision.ps1#L82

maybe not, would probably be simpler if we built an image per java version, otherwise they are all installed and the path needs to be manipulated.

cc @dduportal

for the kubernetes pods / aci it should all work fine as there'll only be one installed

[jglick]

simpler if we built an image per java version, otherwise they are all installed and the path needs to be manipulated

Adjusting the path should be easy enough I guess.

[dduportal]

I got 2 question in order to move forward:

• What is the issues with using tool? (I never used it before on my own controllers but I don't see immediatly the challenge)
• If we adjust the path, is it ok to define the PATH/JAVA_HOME variables at agent level (so we would avoid the complexity of maintining multiple VM images, and we would use the Casc config to define different agent templates with the same VM image but different Java default version)?

[dduportal]

I've started working on this one.
The main challenge is to maintain the "datasource" that will list the java installation paths per Operating System (and per java version).

This datasource can be represented with a 2-levels map such as the following:

def javaSetups = [
      jdk8: [
          windows: 'C:\\Tools\\jdk8u302-b08',
          linux: "/opt/jdk8u302-b08",
      ],
      jdk11: [
          windows: 'C:\\Tools\\jdk-11.0.12+7',
          linux: "/opt/jdk-11.0.12+7/bin/java",
      ],
      jdk17: [
          windows: 'C:\\Tools\\jdk-17+35',
          linux: "/opt/jdk-17+35/bin/java",
      ],
  ]

But it means that we must update this map each time there is a JDK upgrade on the VM templates.

• It's the same challenge as today's tool though
• It should be automatable through our current process that checks for "latest template version" to update the ci.jenkins.io's template IDs

=> What do you think about keep using the "tool" setup but disable the automatic download and use the JAVA_HOME path instead (in the tool configuration) ?

• CasC managed so the update can be synchronized with agent template's update on a single PR
• can be overriden per controller (not only ci.jenkins.io uses this shared library)
• no need to update the shared library groovy code on each JDK upgrade
• The tool already defines tools per platform (based on agent labels) so it already holds the datasource for us and it should avoid messing with PATH names (slash? antislash, etc.)

WDYT @timja @jglick ?

[timja]

I would just extract it out to a major version level

/opt/java-8
/opt/java-11
/opt/java-17

We don't need the minor version in the path, installers don't include it.

[jglick]

What do you think about keep using the "tool" setup but disable the automatic download

That could work, if you can somehow arrange for each VM agent to get a https://javadoc.jenkins.io/hudson/tools/ToolLocationNodeProperty.html with suitable configuration.

I think it would be simpler, though, to just use generic permanent paths like /opt/jdk-8 and hard-code those into the library. No need to mess with CasC.

[dduportal]

Gotcha. Started jenkins-infra/packer-images#103 to have a set of static paths on the VMs to get started.

Next steps:

• Same for container images
• Define tools with the static path (still worth it to avoid unneeded downloads from untrusted sources)
• Define static paths on the library here

[timja]

not sure if needed for container images as they will only have one java version installed.

as we take them (mostly) from upstream.

[dduportal]

Wip in #231

[dduportal]

Fixed by #231