nodejs / ci-config-travis Goto Github PK

Original point raised in #6 (comment) (and subsequent discussion in the PR)

Strictly speaking, even though the general recommendation is to always run the latest version of Node.js in production, and even though older versions in any Node.js major release are considered unsupported, the declaration of "engines": ">= N" in package.json implies that the package should work in N.0.0.

This means that people should test in N.0.0 to avoid accidentally breaking downstream users by using features available only in the latest releases and we could possibly try to find a way to provide such a configuration.

Some discussion points:

• Should we evangelize this to be the default in the ecosystem (and move on to include N.0.0 in future gte-N.yml files)?
• Should we do it at all? Some package maintainers are likely to just bump major (as they should) when they introduce breaking changes (new minimum Node.js version requirement) and move on. This means that they should likely manually update their build matrix to include the minimum supported version explicitly - it is infeasible to provide all possible minimum version configurations.
• Does the "all minors" config (#10) solve the issue well enough to not bother with a separate config?
• Similar decision needed in #10 - should we use a subfolder or a file suffix for this configuration?
• Naming (be it a new strategy, or a folder, or a suffix)
• Do we need to include the N.0.0 for each major release line in the matrix? Can we get away with just one extra build job with the very earliest N.0.0 (which by definition will break if you try to use features from (N+1).0.0)?
• Does semver need a way to define a range which asks for a latest (known) version, rather than all versions greater than N? See also: nodejs/package-maintenance#393

Initial discussion: #6 (comment)

At the very least, there should be something, which can verify whether all the Travis imports are parseable and validate on Travis while still in PR stage.

We are going through all of the node.js repositories to rename the primary branch to main. Please see nodejs/node#33864 for more context.

I don't think there should be any issues for the contributors to this repo. I'd like to do the remain this Friday May 22nd unless there are any concerns.

Please let me know in this issue before May 22nd if there are any concerns, otherwise I'll go ahead and do the change.

@dominykas

Background: the intent of this repo is to provide a set of shared configurations that package maintainers can use in their .travis.yml to automatically add new Node.js versions to their test matrix.

Here's my proposal for the repo layout:

• [strategy]/ (see below)
  • README.md (explain the policy/strategy, include a table of when a new version would be added into each config [and dropped in case of "support LTS, but also test in current" policy], add some examples)
  • gte-N.yml (major versions, starting with N, not sure what minimum N do we want to have when we start this - 0.8? 4? 8? 10?)

I'm thinking we need to support the following strategies:

• all
  • support levels: all, active, supported, current
  • adds a new major as soon as it is released
  • never removes majors
• lts
  • support levels: lts, lts_active, lts_latest
  • adds a new major as soon as it is released
  • removes non-LTS majors as soon as they stop being supported; this is OK, as there is no support for these versions - but the recommendation is to still use them for testing (to spot regressions early)
• lts/strict (I'm thinking of hiding this in a subfolder, to nudge people towards the "support LTS, but also test in current" approach)
  • adds a new major when it enters LTS
  • never removes majors

Possible extensions:

• Include minor versions - not sure this is necessary for the initial version? But it could be added as [strategy]/minors sub-folder or gte-N-minors.yml suffix? (I think I favor the sub-folder)
• An in-between strategy for LTS - only add majors which will become LTS eventually, but do it as soon as they are released

Some guarantees we should provide:

• Once a version is added into the matrix, it is never removed (except for the strategy where it explicitly only adds "current" for testing purposes).
• Once an include file is created, it is never removed

Other observations:

• Versioning this does not make sense - there's only master, it is never breaking; if we need to break it - I'm thinking we should restart in a brand new repo?
• Unsure whether the tooling to generate the ymls needs to live in this repo or separately? I'm leaning towards keeping it separately, but that might be harder for folks who need support to follow?

This came out of a discussion during nodejs/package-maintenance#379 meeting.

We've agreed in pkgjs that we recommend to bump semver major when dropping support for a specific node version, so that's why the initial proposal in #5 does not automatically remove the versions that people test in.

That said, part of the intent of this repository is to reduce the maintenance burden and so changing the gte-N import could be considered a minor hassle. We might want to offer a config which you simply import, and it automatically drops the node versions that are no longer maintained (possibly with a delay?).

I know this could be a contentious discussion, so I'm splitting this out of #5.

Some folks would likely want to have a larger test matrix based on their version support needs, at the cost of slower builds / more build minutes.

As suggested in #5, this could be done as a sub-folder inside a strategy folder, or it can be done as a suffix, e.g. gte-N-minors.yml.

This could somewhat address the "people should test in N.0.0" case (for packages that declare their engines support as >= N / N.x), to avoid accidentally breaking downstream users by using features available only in the latest releases (points raised in #6 (comment) and the remaining discussion).