jenkins-infra / pipeline-library

Collection of custom steps and variables for our Jenkins instance(s)

Pipeline Global Library for ci.jenkins.io

This repository contains a series of steps and variables for use inside of the Jenkins project’s own Jenkins instance(s).

Check this description of available services.

Useful steps:

buildPlugin

Applies the appropriate defaults for building a Maven-based plugin project on Linux and Windows.

You are advised to be using a 2.x or newer parent POM.

Jenkinsfile

buildPlugin(
  forkCount: '1C', // run this number of tests in parallel for faster feedback.  If the number terminates with a 'C', the value will be multiplied by the number of available CPU cores
  useContainerAgent: true, // Set to `false` if you need to use Docker for containerized tests
  configurations: [
    [platform: 'linux', jdk: 21],
    [platform: 'windows', jdk: 17],
])

Optional arguments

• repo (default: null inherit from Multibranch) - custom Git repository to check out

• useContainerAgent (default: false) - uses a Container agent instead of a Virtual Machine: usually faster to start and generates less costs for the project

  • Please note that the implementation of "containers" can be changed over time

• forkCount (default: null) - Execute tests with forkCount Java virtual machines. If no value is specified, a single JVM is used. Common values include 0.45C, 1, 2, and 1C. If you terminate the value with a 'C', that value will be multiplied by the number of available CPU cores. This controls the number of JVM’s used for the test code. Additional JVM’s will be launched if the test code starts agents or uses RealJenkinsRule.

• failFast (default: true) - instruct Maven tests to fail fast

• platforms (default: ['linux', 'windows']) - Labels matching platforms to execute the steps against in parallel

• jdkVersions (default: [8]) - JDK version numbers, must match a version number jdk tool installed

• jenkinsVersions: (default: [null]) - a matrix of Jenkins baseline versions to build/test against in parallel (null means default, only available for Maven projects)

• configurations: An alternative way to specify platforms, jdkVersions and jenkinsVersions (that can not be combined with any of them).

• useArtifactCachingProxy: (default: true) - if set to false, artifacts will not use one of the artifact caching proxy depending on the agent provider (Azure, DigitalOcean or AWS), and will directly use repo.jenkins-ci.org instead (discouraged as less reliable and consuming bandwidth)

  • Those options will run the build for all combinations of their values. While that is desirable in many cases, configurations permit to provide a specific combinations of label and java/jenkins versions to use

    buildPlugin(/*...*/, configurations: [
      [ platform: "linux", jdk: "17", jenkins: null ],
      [ platform: "windows", jdk: "17", jenkins: null ],
      [ platform: "linux", jdk: "21", jenkins: "2.436" ]
    ])

• tests: (default: null) - a map of parameters to run tests during the build. The test results and the JaCoCo code coverage results are recorded after the build with the corresponding Jenkins plugins.

  • skip - If true, skip all the tests by setting the -skipTests profile. It will also skip FindBugs in modern Plugin POMs.

• jacoco: (default: null) - a map of parameters to change the default configuration of the recordCoverage step of the Code Coverage Plugin. This step is called after a plugin build to record the code coverage results of JaCoCo. See recordCoverage step documentation for a list of available configuration parameters.

• pit: (default: null) - a map of parameters to change the default configuration of the recordCoverage step of the Code Coverage Plugin. See recordCoverage step documentation for a list of available configuration parameters. Since running PIT is a time-consuming task, PIT is disabled by default. You need to enable it by setting the property skip to false as well.

• spotbugs, checkstyle, pmd, cpd: (default: null) - a map of parameters to archive SpotBugs, CheckStyle, PMD, or CPD warnings, respectively (only available for Maven projects). These values can replace or amend the default configuration for the recordIssues step of the Warnings NG Plugin. See Warnings NG Plugin documentation for a list of available configuration parameters.

• timeout: (default: 60) - the number of minutes for build timeout, cannot be bigger than 180, i.e. 3 hours.

Note	The recordIssues steps of the warnings plugin and the recordCoverage steps of the coverage plugin run on the first platform/jdkVersion,jenkinsVersion combination only. So in the example below it will run for linux/jdk11 but not on jdk17.

Usage:

Jenkinsfile

buildPlugin(platforms: ['linux'],
        jdkVersions: [11, 17],
        jacoco: [sourceCodeRetention: 'MODIFIED'],
        pit: [skip: false],
        checkstyle: [qualityGates: [[threshold: 1, type: 'NEW', unstable: true]]],
        pmd: [trendChartType: 'TOOLS_ONLY', qualityGates: [[threshold: 1, type: 'NEW', unstable: true]]])

buildPluginWithGradle()

Builds a Jenkins plugin using Gradle. The implementation follows the standard build/test/archive pattern. The method targets compatibility with Gradle JPI Plugin, and it may not work for other use-cases.

Optional arguments

• repo (default: null inherit from Multibranch) - custom Git repository to check out

• failFast (default: true) - instruct the build to fail fast when one of the configurations fail

• platforms (default: ['linux', 'windows']) - Labels matching platforms to execute the steps against in parallel

• jdkVersions (default: [8]) - JDK version numbers, must match a version number jdk tool installed

• configurations: An alternative way to specify platforms, jdkVersions (that can not be combined with any of them)

  • Those options will run the build for all combinations of their values. While that is desirable in many cases, configurations permit to provide a specific combinations of label and java/jenkins versions to use

    buildPluginWithGradle(/*...*/, configurations: [
      [ platform: "linux", jdk: "8" ],
      [ platform: "windows", jdk: "8"],
    ])

• tests: (default: null) - a map of parameters to run tests during the build

  • skip - If true, skip all the tests.

• jacoco: (default: null) - a map of parameters to change the default configuration of the recordCoverage step of the Code Coverage Plugin. This step is called after a plugin build to record the code coverage results of JaCoCo. See recordCoverage step documentation for a list of available configuration parameters.

• spotbugs, checkstyle: (default: null) - a map of parameters to archive SpotBugs or CheckStyle warnings, respectively. These values can replace or amend the default configuration for the recordIssues step of the Warnings NG Plugin. See Warnings NG Plugin documentation for a list of available configuration parameters.

• timeout: (default: 60) - the number of minutes for build timeout, cannot be bigger than 180, i.e. 3 hours.

• noIncrementals: (default: false) - de-activates incremental version publication.

Limitations

Not all features of buildPlugin() for Maven are supported in the gradle flow. Examples of not supported features:

• Configuring jenkinsVersion for the build flow (as standalone arguments or as configurations)

• Usage of Azure Container Instances as agents (only Maven agents are configured)

infra.isTrusted()

Determine whether the Pipeline is executing in an internal "trusted" Jenkins environment

Jenkinsfile

if (infra.isTrusted()) {
    /* perform some trusted action like a deployment */
}

infra.ensureInNode(nodeLabels, body)

Ensures that the given code block is runs in a node with the specified labels

Jenkinsfile

infra.ensureInNode('docker,java') {
    sh 'docker -v'
}

runBenchmarks

Runs JMH benchmarks and archives benchmark reports on highmem nodes.

Supported parameters:

artifacts

(Optional) If artifacts is not null, invokes archiveArtifacts with the given string value.

Example

runBenchmarks('jmh-report.json')

buildDockerAndPublishImage(imageName, config)

Lints, Builds, then publishes a docker image.

Adds a bunch of build args you can use in your docker image:

• GIT_COMMIT_REV - The commit that triggered this build

• GIT_SCM_URL - Url to repo

• BUILD_DATE - Date that the image was built (now)

Supported parameters:

imageName

Name of the docker image to build

config

(Optional) map of extra flags

• agentLabels: String expression for the labels the agent must match

• automaticSemanticVersioning: Do not automagically increase semantic version by default

• includeImageNameInTag: Set to true for multiple semversioned images built in parallel, will include the image name in tag to avoid conflict

• dockerfile: override the default dockerfile of Dockerfile

• targetplatforms: defined the platforms to build as TARGET

• nextVersionCommand: Commmand line used to retrieve the next version (default 'jx-release-version')

• gitCredentials: override Credential ID for tagging and creating release

• imageDir: Relative path to the context directory for the Docker build

• registryNamespace: empty = autodiscover based on the current controller, but can override the smart default of jenkinsciinfra/ or jenkins4eval/

• unstash: Allow to unstash files if not empty

• dockerBakeFile: Allow to build from a bake file instead

• dockerBakeTarget: Allow to specify a docker bake target other than 'default'

• disablePublication: (Optional, default to false) Allow to disable tagging and publication of container image and GitHub release

Example

buildDockerAndPublishImage('plugins-site-api')
buildDockerAndPublishImage('inbound-agent-maven:jdk8-nanoserver', [
      dockerfile: 'maven/jdk8/Dockerfile.nanoserver',
      agentLabels: 'docker-windows-2019 && amd64',
      targetplatforms: 'windows/amd64',
      imageDir: 'maven/jdk8',
    ])

is also called from parallelDockerUpdatecli with config within buildDockerConfig like this :

parallelDockerUpdatecli([
  imageName: 'wiki',
  rebuildImageOnPeriodicJob: false,
  updatecliConfig: [containerMemory: '1G'],
  buildDockerConfig : [targetplatforms: 'linux/amd64,linux/arm64,linux/s390x']
])

Contribute

Requirements

• (Open)JDK v8

• Maven 3.6.x

Testing a pull request

By adding @Library('pipeline-library@pull/<your-pr-number>/head') _ at the top of a Jenkinsfile from a repository built on one of the *.ci.jenkins.io instances, you can test your pipeline library pull request on ci.jenkins.io.

A repository is dedicated for these kind of tests: https://github.com/jenkinsci/jenkins-infra-test-plugin/

pipeline-library's Issues

Pipeline succeeds with failure during incremental upload

Hi. When publishing to https://incrementals.jenkins.io/ it sometimes happen that the artifact couldn't be uploaded for any reason. The pipeline build then does not fail or at least become unstable.

Unfortunately, I haven't noticed that in the Gitea plugin for my PR and it got merged. Now the main branch cannot upload the incremental build either.

Is there any reason to have the valid response codes including non 2xx codes?

pipeline-library/vars/infra.groovy

Line 322 in 18bf46d

validResponseCodes: '100:599',

Library build fails on Java 17

Is there a special environment required to get a green build locally?

When I run mvn --no-transfer-progress -B clean verify

I get

Apache Maven 3.9.1 (2e178502fcdbffc201671fb2537d0cb4b4cc58f8)23
Maven home: /usr/local/Cellar/maven/3.9.1/libexec
Java version: 17.0.6, vendor: Homebrew, runtime: /usr/local/Cellar/openjdk@17/17.0.6/libexec/openjdk.jdk/Contents/Home
Default locale: de_DE, platform encoding: UTF-8
OS name: "mac os x", version: "13.3.1", arch: "x86_64", family: "mac"
 [ ...]
[ERROR] Failures: 
[ERROR]   BuildPluginStepTests.test_getConfigurations_implicit_with_jdkVersions:107 expected:<[null, null, null, null]> but was:<4>
[ERROR]   BuildPluginStepTests.test_getConfigurations_implicit_with_jenkinsVersions:96 expected:<[null, null, null, null]> but was:<4>
[ERROR]   BuildPluginStepTests.test_getConfigurations_implicit_with_platforms:85 expected:<[null, null]> but was:<2>
[ERROR]   LaunchableStepTests.test_launchable_install:21
[INFO] 
[ERROR] Tests run: 106, Failures: 4, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

buildDockerAndPublishImage: manage "monorepo" with a new `multipleSemverImagesBuild` parameter

For "monorepo" like docker-inbound-agents, this multipleSemverImagesBuild new parameter (default: false) would allow building multiple images with semver activated in parallel by including the image name in the tag to avoid conflict between images, while keeping retro-compatibility for repository building only one image.

Since we're dealing with these versions with Updatecli, there is no problem for us stripping the image name from it.

Follow-up of #395

Custom Maven properties

What feature do you want to see added?

In https://github.com/jenkinsci/winp I would like to have a Matrix build where one configuration uses native.configuration=Release and another uses native.configuration=Debug. Right now I can customize jenkins.version per parallel branch, but not other Maven properties. It would nice to be able to pass in arbitrary Maven properties. I could of course copy and paste the entire buildPlugin() step into that repository, but that would not be good from the perspective of code reuse.

Upstream changes

No response

Plugin does not build on Jenkins

I added your sources to my repo, created a multibranch pipeline on Jenkins and ran it. Unfortunately, I get a some test errors and I don't know how to resolve them.

Is there any guide on how to create your plugin or maybe just a link to the hpi file? As far as I could see, it's not available via the plugin center of Jenkins. Actually, I need this plugin only to build another plugin.

This is the error message on Jenkins 2.319.2 (latest LTS):
[ERROR] Failures:
[ERROR] BuildPluginStepTests.test_getConfigurations_implicit_with_jdkVersions:102 expected:<[null, null, null, null]> but was:<4>
[ERROR] BuildPluginStepTests.test_getConfigurations_implicit_with_jenkinsVersions:91 expected:<[null, null, null, null]> but was:<4>
[ERROR] BuildPluginStepTests.test_getConfigurations_implicit_with_platforms:80 expected:<[null, null]> but was:<2>
[INFO]
[ERROR] Tests run: 89, Failures: 3, Errors: 0, Skipped: 1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:03 min
[INFO] Finished at: 2022-01-18T22:19:35Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project pipeline-library: There are test failures.

Default `buildPlugin()` and `buildPluginWithGradle()` to Java 11

Both calls still default to Java 8, when the minimum Java version to compile plugins against modern baselines require Java 11.

It would be nice to have Java 11 available as default, given that's the lowest supported Java version by now.

Usage of deprecated `publishCoverage` step in `buildPlugin()`

According to the pipeline docs, publishCoverage is deprecated.
Yet, this is still called within buildPlugin().

Code coverage should likely be published via recordCoverage calling the jacoco task.

Allow to add a `skip-updatecli` label on pull requests to avoid failing builds when introducing new files

feature request: allow running `updatecli()` in the current agent

By default, the updatecli() library spanws an agent based on a container (easier, faster by default).

As a end user, I would want to execute the updatecli commands in the current agent context instead of spawning an agent.

This RFE could be, instead, the same as updatecli/updatecli-action#76, e.g. instead of trying to spawn an agent, the library should check for the updatecli binary (eventually with the specified version) in the current agent, otherwise installing the updatecli binary for the current context.

Don't fail builds if hadolint logs something on the "info" level

It's useful that the build fails if hadolint finds something on the "warning" level or more severe, but general items emitted on the "info" level cause the build to fail too, isn't that a bit too strict?

`buildDockerAndPublishImage` fails at `GitHub Release` stage if there are more than one "next" draft releases

pipeline-library/vars/buildDockerAndPublishImage.groovy

Line 199 in 8b9941b

final String release = sh(returnStdout: true, script: 'gh api ${GH_RELEASES_API_URI} | jq -e -r \'.[] | select(.draft == true and .name == "next") | .id\'').trim()

Example of failing build :

Corresponding releases

https://github.com/jenkins-infra/docker-openvpn/releases

tune the number of forks run for surefire

the CI nodes may (or may not have a large number of CPUs), and by default we only run one surefire fork for all tests.

This could be a bit wasteful for many plugins where multiple tests run in parallel could hugely decrease the plugins build time.
(e.g. jenkinsci/git-plugin#1173 (comment))

therefore the number of forks should be tuned for all plugins to a reasonable default (we already tune the surefire JVM args to a set JVM heap so this number could reasonably be caclulated knowing the underlying hardware).

Additionally this tuning may not work for all plugins (for example warnings ng or plugins using RealJenkinsRule that require more memory, so allowing this number to be tweaked / overriddien would be required at the same time (or the number chosen would have to be low enough to account for these plguins which could be deemed as wastefull).

Quicker builds on the same hardware will also reduce the number of agents we require which should reduce the cloud costs for the product.

cleanup request: replace all the `readYaml()` occurences by an agent command (`yq`, `sed`, whatever)

The readYaml() feature is used on this repository as a pipelien step provided by https://github.com/jenkinsci/pipeline-utility-steps-plugin/blob/master/docs/STEPS.md.

But this step runs on the controller, wasting precious resources that could better be used on other tasks such as... handlin build orchestration and agents.

The request is to replace the readYaml() occurence by a call to an agent with command line tools such as yq (which is installed on all Linux agents and Windows VM agents).

[buildAndPublishDocker] error when recording lint results when the docker image name contains `:`

The pipeline at https://github.com/jenkins-infra/docker-inbound-agents/blob/main/Jenkinsfile_k8s is failing for the 3 images with maven-jdk(.* images with the errors like:

Could not instantiate {enabledForFailure=true, aggregatingResults=false, tool=@hadoLint(id=inbound-agent-maven:jdk17-hadolint-1642494989957,pattern=inbound-agent-maven:jdk17-hadolint-1642494989957.json)} for io.jenkins.plugins.analysis.core.steps.RecordIssuesStep: java.lang.IllegalArgumentException: Could not instantiate {id=inbound-agent-maven:jdk17-hadolint-1642494989957, pattern=inbound-agent-maven:jdk17-hadolint-1642494989957.json} for io.jenkins.plugins.analysis.warnings.HadoLint: java.lang.reflect.InvocationTargetException.

Once the build is finished, we can see the following message in the java log stack at the end:

java.lang.IllegalArgumentException: An ID must be a valid URL, but 'inbound-agent-maven:jdk8-hadolint-1642493238043

Many thanks for @uhafner to point us to https://github.com/jenkinsci/warnings-ng-plugin/releases/tag/v9.10.3 (we upgraded 6 days ago: jenkins-infra/docker-jenkins-weekly#361).

It means that we should update https://github.com/jenkins-infra/pipeline-library/blob/master/vars/buildDockerAndPublishImage.groovy#L73 to remove the forbidden character such as :.

Extend configuration with `testsCategories` parameter

Hello,

As far as I see, the current implementation of buildPlugin supports defining multiple configurations of environments where build will happen. For example, let's consider Jenkinsfile from docker-workflow-plugin plugin:

buildPlugin(platforms: [
    'linux',
    'maven-windows'
])

Also, I've noticed that the build will be run in a parallel mode where each branch is executed in its separate environment. And seems like the above code can be rewritten in another form:

buildPlugin(configurations: [
    [ platform: 'linux' ],
    [ platform: ''maven-windows' ]
])

And, in my opinion, it will super helpful if we can extend this definition with testsCategories parameter or something similar, for instance:

buildPlugin(configurations: [
    [ platform: 'linux', testsCategories: 'Linux' ],
    [ platform: 'maven-windows', testsCategories: 'Windows' ]
])

What do you think about such an idea? Maybe you already have something similar in your backlog, so just share the link to the corresponding issue.

Stop using `tool`

jenkinsci/aws-global-configuration-plugin#19

Unpacking https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.1/apache-maven-3.8.1-bin.zip to C:\Jenkins\tools\hudson.tasks.Maven_MavenInstallation\mvn on aci-maven-8-windows-c4kkv
Unpacking https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.12+7/OpenJDK11U-jdk_x64_windows_hotspot_11.0.12_7.zip to C:\Jenkins\tools\hudson.model.JDK\jdk11 on aci-maven-8-windows-c4kkv

We should pack the JDK & Maven into VMs, not just container images, or at a minimum offer our own (e.g., Artifactory) download locations rather than using public URLs.

`maven` label broken

As seen for example in jenkinsci/jjwt-api-plugin#35, some recent change (whether to this lib or to underlying infrastructure incl. Docker images) seems to have broken

buildPlugin(platforms: ['maven'])

which now gives this error:

+ mvn … clean install
The JAVA_HOME environment variable is not defined correctly,
this environment variable is needed to run this program.

Related to #228 I guess, in that the build is attempting to use the tool step when it should not. Perhaps that was long true but until recently was harmless?

Use `main` as primary branch

Suggestion

We should use main instead of master as primary branch.

This will need updating not only this repo but also all references to master in Jenkinsfile(s) and job on ci.jenkins.io, infra.ci.jenkins.io, release.ci.jenkins.io and trusted.ci.jenkins.io (not sure about cert.ci.jenkins.io), and changing settings of Jenkins instances around global shared pipeline library.

Links

• jenkins-infra/helpdesk#2671

Feature Request(buildand publichDocker) allow testing images of PRs in the `jenkins4eval` namespace

If you have a PR like jenkins-infra/docker-builder#34, you would want to test the image before merging/releasing.

@halkeye suggested on IRC that jenkins4eval/ could be used for that.

This issues is about updating the library so that the deployment stage is run even on PRs, at least on infra.ci, by pushing the image to jenkins4eval/jenkinsciinfra-: