octue / conventional-commits

Continuous deployment via Conventional Commits

Continuous deployment via Conventional Commits

This repository contains three tools enabling continuous deployment using Conventional Commits - two GitHub actions and a pre-commit hook:

• A semantic version checker that uses git-mkver to predict what the version of the package should be as of the HEAD commit and checks if this matches the version as currently stated in a setup.py, setup.cfg, pyproject.toml, or package.json file.
• A pull request description generator that categorises all the commit messages in a pull request and compiles them into a well-formatted description ready to be used as release notes. These can be used to automatically update a section of the pull request description on every push while leaving the rest of the description as-is.
• A commit-msg-type pre-commit hook that checks if the current commit message adheres to the Conventional Commit standard.

The GitHub actions can be combined with an automatic release-on-pull-request-merge workflow to facilitate continuous deployment of correctly semantically-versioned code changes to your users (as long as all contributers categorise their commits correctly as breaking changes, new features, and bug-fixes/small-changes). Examples of workflows that do this are linked below. You can find an example release workflow here.

Contents

• Commit message pre-commit hook
• Semantic version checker - this has now been moved to its own repository
• Pull request description generator - this has now been moved to its own repository

Conventional commit message pre-commit hook

Description

A commit-msg-type pre-commit hook that checks whether each commit message adheres to the Conventional Commits standard, as well as the additional customisable rules that:

• The header:
  • Uses only the allowed commit codes
  • Is no longer than the maximum header length
  • Ends in a valid pattern
• The body:
  • Is present if required
  • Has lines no longer than the maximum body line length

You can provide values for each of these rules, including another set of commit codes to override or augment the defaults.

Default allowed commit codes

You can change these for a repository (see 'Usage' below) for your requirements. Across octue, we use:

• FEA: A new feature
• ENH: An improvement or optimisation to an existing feature
• FIX: A bug fix
• OPS: An operational/devops/git change e.g. to continuous integration scripts or GitHub templates
• DEP: A change in dependencies
• REF: A refactor of existing code
• TST: A change to tests or the testing framework
• MRG: A merge commit
• REV: A reversion e.g. a git revert commit
• CHO: A chore e.g. updating a menial configuration file or .gitignore file
• WIP: A work-in-progress commit (usually to be avoided, but makes sense for e.g. trying changes in git-based CI)
• DOC: A change to documentation, docstrings, or documentation generation
• STY: A change to code style specifications or to code to conform to new style

Usage

Use this hook in your repository by adding it to your .pre-commit-config.yaml file as:

- repo: https://github.com/octue/conventional-commits
  rev: 0.0.2 # (or another version)
  hooks:
    - id: check-commit-message-is-conventional
      stages: [commit-msg]
      args:
        - --additional-commit-codes=ABC,DEF,GHI
        - --maximum-header-length=72
        - --valid-header-ending-pattern=[A-Za-z\d]
        - --require-body=0
        - --maximum-body-line-length=72

Then, install pre-commit if you haven't already:

pip install pre-commit

Finally, install the hook:

pre-commit install && pre-commit install -t commit-msg

Divergence from Conventional Commits specification

Note that while this hook complies with nearly all of the Conventional Commits specification, it is diverges slightly in the following ways:

• Scopes are disallowed (scopes are an optional part of the specification) for readability and consistency
• FEA is used instead of feat
• Every extra commit code we have added to the default set consists of three capital letters. This means that commit codes (type prefixes) always line up in git log --oneline view for ease of viewing and mental (human) parsing. We require that they are always provided in uppercase in commit headers, again to increase ease of viewing. Despite this, you can add your own codes to this default set that are in whatever form you like (e.g. any number of letters in lowercase).
• Footers are not validated against the specification
• Breaking changes are validated but are allowed to appear in the body as well as the footer

Readability gains of 3-letter uppercase commit codes/types

Only using 3-letter uppercase commit codes/types results in a uniform, easily readable git log. There is a clear distinction between the code and the title of the commit, and the eye doesn't have to jump left and right on each new line to find the start of the title. Here is an example from our own git log:

>>> git log --oneline -10

82f5953 ENH: Validate breaking change indicators in commit messages  (HEAD -> main)
810944a ENH: Improve range of commit codes available
311f4f5 REF: Move comment removal into method  (origin/main, origin/HEAD)
ba2aca3 IMP: Explain commit codes in error message
f0142c2 DOC: Update README
214af4f TST: Test optional CLI args
417efcc IMP: Add DOC and LOG commit codes
d528edd OPS: Use version of hook specified in this repo locally
5b5727c IMP: Allow options to be passed to hook
86e07c5 CLN: Apply pre-commit checks to all files

conventional-commits's Issues

Release notes not generating on GitHub on some repositories but are locally

Bug report

What is the current behavior?

On this repository, the release notes generator is producing empty release notes when the GitHub workflow is run but produces the expected release notes when the same code is run locally on the repo

What is the expected behavior?

The same release notes should be produced on the GitHub workflow as are locally.

Ignore commit header line length limit if commit code is `MRG`

Feature request

Use Case

Using git merge usually results in a long commit header which is inconvenient to reduce to below the header line limit. We should add an option to ignore the line length limit for these commits, perhaps with the default being to ignore it.

Add option to add link to relevant pull request in release notes

Link release note items to commit diff

Pretty-Print available versions

Feature request

Use Case

I want to be able to easily read the output of the hook message

Current state

When using the hook and I add a noncompliant commit message, I get output that looks like this:

Ideally, it'd look like:

COMMIT MESSAGE FAILED CHECKS: Commit headers should start with one of the allowed commit codes...

    'FEA:' A new feature
    'ENH:' An improvement or optimisation to an existing feature
    'FIX:' A bug fix
    'OPS:' An operational/devops/git change e.g. to continuous integration scripts or GitHub templates
    'DEP:' A change in dependencies
    'REF:' A refactor of existing code
    'TST:' A change to tests or the testing framework
    'MRG:' A merge commit
    'REV:' A reversion e.g. a `git revert` commit
    'CHO:' A chore e.g. updating a menial configuration file or .gitignore file
    'WIP:' work-in-progress commit (usually to be avoided, but makes sense for e.g. trying changes in git-based CI)
    'DOC:' A change to documentation, docstrings, or documentation generation
    'STY:' A change to code style specifications or to code to conform to new style

...received 'DOC  Added note on how to add fixtures'.

Always compares to main branch

Bug report

What is the current behavior?

If I have branches

main -> dev -> myfeature

And I use

on: pull_request

in my GHA, and make a pull request into a dev from myfeature, the contents of myfeature are compared to the contents of main, not to the contents of dev, so the PR summary contains all of the work in main.

What is the expected behavior?

It should compare to the base branch, not the main branch.

Raise an error if mkver.conf isn't present

I want a boom

Feature request

Implement a set of Github Actions

Feature request

Use Case

We currently have a lot of complex scripts hanging around that we embed into github actions, for example:

      - name: Get Version
         id: get-version
          run: |
            PACKAGE_VERSION=$(node -p -e "require('./package.json').version")
            echo "::set-output name=version::${PACKAGE_VERSION}"

        - name: Convert branch name to tag
          # Get the github ref as a slugified docker tag
          id: branch
          run: echo "::set-output name=branch_tag::$(echo ${GITHUB_REF#refs/heads/} | iconv -c -t ascii//TRANSLIT | sed -E 's/[~^]+//g' | sed -E 's/[^a-zA-Z0-9]+/-/g' | sed -E 's/^-+|-+$//g' | tr A-Z a-z)"

      - name: Check Version
        if: # If it's a PR into main...
        run: |
          something complicated

It would be good to begin compiling a set of github actions to streamline this process in repositories using conventional commits (and octue in general) so we could define workflows that look like:

      - name: Get Version
        uses: octue/actions-get-version@v1 # or gha-get-version or something consistent
        from: package.json

      - name: Check Version
        if: # If it's a PR into main, otherwise skip check
        uses: octue/actions-check-version@v1

Add whitespace to valid commit header endings

A strict rule against whitespace at the end of commit headers is frustrating to the user and doesn't add anything to the quality of the commit message

Allow quotation marks at end of commit headers

These are used by default by git for e.g. git revert

Add support for `DEPRECATION` notes in commits

This would be similar to the existing support for BREAKING CHANGE notes and would allow deprecations to be automatically flagged in PR/release notes.

Mkver comparison incorrect where package.json not in repository root

Bug report

What is the current behavior?

I'm using a subpackage in a repository which is versioned on a non-standard file path; i.e. packages/components/package.json. There is also a package.json in the root directory (with no version information).

I have a main branch in which the version is presently 0.11.0 and a pull request into that, with several commits, whose version is still 0.11.0.

The semantic version check is passing, and stating that the version 0.11.0 is as expected (i.e. it is reading the correct file to determine the current version).

However the presence of additional patch commits suggests it should be 0.11.1.

Reproduction is at this point in the tree for this PR.

What is the expected behavior?

The checker should fail and indicate that I should be at 0.11.1. I suspect that it's comparing

• [branch] packages/components/package.json to [main] package.json instead of comparing
• [branch] packages/components/package.json to [main] packages/components/package.json

Other information

Now that I think about it, I'm not sure it's possible to even support independently versioned subpackages at all. Perhaps we need to simply accept that the version must be dictated and if a top-level package.json exists that the version be specified in there.

To be discussed.

Consider using ❌ or similar emoji to denote breaking changes

This might make release notes a bit more readable and make it easier for breaking changes to stand out. It can be a bit hard to see them currently.

Add a description of how to document breaking changes

Feature request

Use Case

It's not clear how to document breaking changes, please add a section to the README describing how to do that and showing a screenshot of the resulting generated notes

TypeError on a PR with many commits

Bug report

What is the current behaviour?

In this PR I have hundreds of commits.

I'm guessing that the number of commits is the problem, I'm not certain.

On running the update-pull-request action I get the following error:

Perhaps over a certain number, the commit history is paginated meaning you don't get a string?

Note also that the stage where the error occurs is not the stage where it manifests: The error happens in the previous supposedly successful step

What is the expected behaviour?

• The release notes compiler should exit with a non-zero code if it fails, so that the error is targeted to the right place
• The release notes compiler should work with a long list of commits

To reproduce

Use either that particular PR, or take any repository with a long list of commits, look way back in the history, and check out a branch from there. Then make a PR from main into that branch and run the updater on it.

Document semantic version checker / provide examples

Feature request

Use Case

Since we're now using the version checker in github actions all over the place, it'd be good if there was some documentation around it, even if only a couple of sections in the readme. This is particularly useful for the functionality we don't use in this package itself, since there are no examples available that I can just cut and paste into place.

For example, knowing that check-semantic-version package.json --file packages/components/package.json is how to point to a specific file to do version checking is impossible without some inside knowledge from @cortadocodes or
quite deeply introspecting the code.

Ideally if the documentation pointed to a set of github action examples that people could lift, that's be great.

Add style subheading to release notes

Extra colons interfere with commit classfications

Bug report

What is the current behavior?

A commit message with multiple colons uses the second one to split the commit type.

e.g. OPS: My message: something has commit type OPS: My message, not OPS and is therefore not classified

What is the expected behavior?

Commit messages containing additional colons should not interfere with the conventional commit system.

Semver checker fails due to branch name inclusion

Bug report

What is the current behavior?

Semantic version checker fails because it thinks the version should have the branch and sha appended.

In this check run in which branch feature/add-gw-nordex-ge-turbines is being merged into main the failure is as follows:

The current version (0.1.0) is different from the expected semantic version (0.1.0+feature-add-gw-nordex-ge-turbines.e5a8956).

What is the expected behavior?

With the version at 0.1.0 I expected that to pass correctly.

Remove old scripts

It's been a while since I moved the two GitHub actions in this repo into their own repos. We should remove the deprecated scripts from this repository now.

Allow semver checking against a monorepo package or git submodule

Feature request

Use Case

Sometimes in a repository, you have a folder structure that's like:

packages / packagename / package.json

We want to be able to override the default location of package.json in such a case.

Current state

Currently getting errors because the top level drectory doesn't have a package.json.

Suggested solution

Allow us to specify the location of package.json in the check-semantic-version cli:

jobs:
  check-semantic-version:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          # Set fetch-depth to 0 to fetch all tags (necessary for git-mkver to determine the correct semantic version).
          fetch-depth: 0
      - uses: actions/setup-python@v2
      - name: Install git-mkver
        run: |
          curl -L https://github.com/idc101/git-mkver/releases/download/v1.2.1/git-mkver-linux-amd64-1.2.1.tar.gz \
          | tar xvz \
          && sudo mv git-mkver /usr/local/bin
      - name: Install semantic version checker
        run: pip install git+https://github.com/octue/conventional-commits
      - name: Check version
        run: check-semantic-version npm --file packages/mypackage/package.json

update-pull-request action failing

Bug report

What is the current behavior?

When the update pull request hook fires, it fails like this.

Your environment

Github actions

Consider merging some of the release notes sections

e.g.

• Features and enhancements
• Refactoring, testing, other etc.

Unhide the default prefix list in the readme

Feature request

Use Case

I'm pointing developers to the conventional-commits readme so they can see what the commit codes are.

Current state

I want them to see the list of supported prefixes but they're hidden in a bullet point, so I have to be like "they are there, look down for the triangle and click expand".

The reference for what the commit codes are and how they should be used will be needed far more often than the installation instructions - we've hidden the most important bit!

Support commits with multi-line bodies

Commits with multi-line bodies are currently ignored. These should be supported just like any other commit.

Add breaking change commit notes to release notes

This could be in an indented section under the bullet point for the commit

Consider putting all breaking changes at top of their sections in release notes generator

Conventional Commits

Feature request

Use Case

We want to be able to use conventional commits in our projects to reduce the chore burden of creating releases.

Current state

We have a release workflow that's based on release/x.y.z branches, and automatically creates tagged releases on successful merge of a PR. It works pretty well but has the following flaws:

• We manually have to track any breaking changes and decide on version ourselves. We're a bit fast and loose with breaking things, to be honest.
• Where we make a PR feature/issue-whatever -> release/x.y.z, we have to manually copy the contents of the PR notes across into the relevant sections PR release/x.y.z -> main
• In this release flow, the release numbers are determined a-priori. However, the correct release number is a function of the contents of the release. Adding a single change therefore means constructing a release branch with a new version number, and corresponding with colleagues to make sure nobody's about to release something semver-earlier than that. It's a grind, and violates the core principle of CD, because it tends us to group new features together into a release branch instead of getting them out of the door quickly.
  • With twined and octue-sdk, releases are fairly well controlled and planned. We know what's going into them. However, in amy and windquest (applications, rather than libraries), development is much less structured so the release flow is really painful. Also, because there's a 'staging' step of manual QC before an application goes out of the door, there's an extra grind of copying all the release notes through the staging branch and manually creating a release once they hit main.

Proposed solution

Develop four tools (the following list of conventional commit plugins will be helpful)...

• DO FIRST, get this deployed across all repos Implement a pre-commit hook in this repository to enforce commit message patterns. Can be inspired by this and/or follow the guide at pre-commit.com.
• DO SECOND Implement a GHA, that can be run on pull requests to check that the version (e.g. in package.json is correct) and, optionally, update and commit the correct value automatically (can use, for example, this tool)
• DO THIRD Find or implement a GHA that can be run on PRs to update their Notes with sections on new features, breaking changes and minor fixes automatically from the commit contents
• DO FOURTH Find or implement a GHA that can be run on merge of code into main to create an automated release from the PR notes. We already have this, but it might need tweaking - it currently checks the version against the release/x.y.z branch name. It needs to check the version against the calculated value instead.

Indicate in release notes what breaking changes have taken place

Write article on the benefits of conventional-commit-based continuous deployment on Medium for the new website

Test rest of package

Strange merge commits always appear in release notes

Bug report

What is the current behavior?

There is always an entry in the uncategorised section when release notes are autogenerated e.g.

### Uncategorised!
- [x] Merge b3b7e4d into 9c125ab

What is the expected behavior?

This shouldn't be there.

Remove line length limit for commit bodies or format it away so upgrade instructions render nicely in markdown

Add commit codes for deprecation and subsequent removal

@thclark what do you think about having e.g. DPR for deprecation and REM for removal of previously deprecated code?

Additionally, should breaking change alerts be attached to releases that:

1. Deprecate code (the code should still work with the same interfaces etc. but the old use will be discouraged)
2. Remove previously deprecated code (the code will no longer work with the old interfaces at all)

Missing commits from PR history

Bug report

What is the current behavior?

There's a problem where many commits are missing from the pull request notes here.

Not sure why the missing commits are missing. Tried merging main (although I was already ahead of the tip) - no difference

What is the expected behavior?

All commits present

Allow brackets at end of commit message

Improve upgrade instructions format in release notes generator

@thclark says:

We need to talk about a convention/pattern for the upgrade instructions, because they need to be there but are too verbose as-is. Perhaps we could put them into <details> sections, where the first line of the breaking change note is the summary line?

BREAKING-CHANGES has a mismatched format

Bug report

What is the current behavior?

I get the following error:

COMMIT MESSAGE FAILED CHECKS: Breaking changes must be denoted by one of {'BREAKING-CHANGE', 'BREAKING CHANGE'} in uppercase followed by ': ' and a description e.g. 'BREAKING CHANGE: Change MyPublicClass interface'; received 'BREAKING-CHANGE: Uncategorised commits contain breaking changes including the use of tabulate as a dependency and refactor to use stream rather than list, as well as substantive changes to wrapping and line termination defaults'.

What is the expected behavior?

This commit check should pass, I don't see anything wrong with it?

More Information

The full commit command was:

gc -am "FEA: Set correct version for incoming updates

BREAKING-CHANGE: Uncategorised commits contain breaking changes including the use of tabulate as a dependency and refactor to use stream rather than list, as well as substantive changes to wrapping and line termination defaults"

Add VER: prefix?

Feature request

Use Case

I have a branch with a PR into main, that's failing the semantic version consistency check. So I need to make a commit that changes the version. It's not clear from the existing list what kind of commit that should be.

Current state

I'm currently using CHO:. Perhaps I should be using OPS:?

Suggested solution

Either we add VER: to our list, or we explicitly state in the list of purposes for existing ones what we should use for bumping versions.