inpsyde / reusable-workflows Goto Github PK
View Code? Open in Web Editor NEWReusable workflows for common GitHub Actions jobs.
License: GNU General Public License v2.0
Reusable workflows for common GitHub Actions jobs.
License: GNU General Public License v2.0
Currently, when running the tests-unit-js.yml
workflow, the following error is shown:
Error: Could not resolve a module for a custom reporter.
Module name: github-actions
Push a commit to a repository using the tests-unit-js.yml
action.
The GitHub action runs without issues.
No response
Run ./node_modules/.bin/jest --reporters=default --reporters=github-actions
Error: Could not resolve a module for a custom reporter.
Module name: github-actions
at /home/runner/work/project-name/project-name/node_modules/jest-config/build/normalize.js:558:15
at Array.map (<anonymous>)
at normalizeReporters (/home/runner/work/project-name/project-name/node_modules/jest-config/build/normalize.js:540:33)
at normalize (/home/runner/work/project-name/project-name/node_modules/jest-config/build/normalize.js:633:5)
at readConfig (/home/runner/work/project-name/project-name/node_modules/jest-config/build/index.js:224:68)
at async readConfigs (/home/runner/work/project-name/project-name/node_modules/jest-config/build/index.js:406:26)
at async runCLI (/home/runner/work/project-name/project-name/node_modules/@jest/core/build/cli/index.js:230:59)
at async Object.run (/home/runner/work/project-name/project-name/node_modules/jest/node_modules/jest-cli/build/cli/index.js:163:37)
Error: Process completed with exit code 1.
No response
I know that GitHub Actions still uses Yarn 1 (see e.g. here).
However, should this workflow be run on a system where the user manually installed a newer version,
yarn config set cache-folder
won't work because the settings key changed in Yarn >= 2. There, it's calledcacheFolder
(documentation), and the command would therefore beyarn config set cacheFolder
.
Originally posted by @tyrann0us in #29 (comment)
Current used secret name is too wide, and cannot be overridden.
See here
I suggest to make it configurable
Considered using a secret config USER_TOKEN.
Consider using SSH configuration.
No response
We had now several discussions internally about:
"How to check if PHP code is compatible with PHP Version x.y".
The decision internally was to introduce a new workflow compat-check-php.yml
. This workflow will look like the following:
name: Compatibility check PHP
on:
workflow_call:
inputs:
PHP_MATRIX:
description: Matrix of PHP versions as a JSON formatted object.
default: '["7.4"]'
required: false
type: string
COMPOSER_ARGS:
description: Set of arguments passed to Composer.
default: '--prefer-dist'
required: false
type: string
secrets:
COMPOSER_AUTH_JSON:
description: Authentication for privately hosted packages and repositories as a JSON formatted object.
required: false
ENV_VARS:
description: Additional environment variables as a JSON formatted object.
required: false
jobs:
compatibility-check-php:
timeout-minutes: 5
runs-on: ubuntu-latest
strategy:
matrix:
php-version: ${{ fromJson(inputs.PHP_MATRIX) }}
env:
COMPOSER_AUTH: '${{ secrets.COMPOSER_AUTH_JSON }}'
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-version }}
coverage: none
- name: Set up custom environment variables
env:
ENV_VARS: ${{ secrets.ENV_VARS }}
if: ${{ env.ENV_VARS }}
uses: actions/github-script@v6
with:
script: |
JSON
.parse(process.env.ENV_VARS)
.forEach(envVar => core.exportVariable(envVar.name, envVar.value));
- name: Install Composer dependencies
uses: ramsey/composer-install@v1
with:
composer-options: ${{ inputs.COMPOSER_ARGS }}
- name: Run PHP compat check
run: find ./src/ ./inc/ ./tests/ -type f -name '*.php' -print0 | xargs -0 -L 1 -P 4 -- php -l
The "Allow build to standalone branches" approaches introduced in #76 added two new inputs to the "Build and push assets" workflow:
BUILT_BRANCH_SUFFIX
RELEASE_BRANCH_NAME
They can be used to define a "suffix to calculate the target branch for pushing assets on the branch
event" and specify the "target branch where compiled assets are pushed and the tag is moved to", respectively.
However, according to a search on GitHub (https://github.com/search?q=org%3Ainpsyde+OR+org%3AInpsyde-Global-Service-Provider+language%3AYAML+inpsyde%2Freusable-workflows%2F.github%2Fworkflows%2Fbuild-and-push-assets.yml%40main+BUILT_BRANCH_SUFFIX&type=code and https://github.com/search?q=org%3Ainpsyde+OR+org%3AInpsyde-Global-Service-Provider+language%3AYAML+inpsyde%2Freusable-workflows%2F.github%2Fworkflows%2Fbuild-and-push-assets.yml%40main+RELEASE_BRANCH_NAME&type=code), these two inputs are only ever used with the values mentioned in the documentation (https://github.com/inpsyde/reusable-workflows/blob/main/docs/build-and-push-assets.md#simple-usage-example). This is understandable, because other values don't make much sense.
It would be better to make the "push to build branch" approach an on/off toggle by introducing a new boolean input, e.g., PUSH_TO_BUILT_BRANCH
, USE_BUILT_BRANCH
or similar, which defaults to false
.
BUILT_BRANCH_SUFFIX
and RELEASE_BRANCH_NAME
should be deprecated and show a deprecation warning. When either of them is used, they should enable the toggle.
Keep things as-is.
Originally posted by @tyrann0us in #115 (review)
It might make sense to make this a boolean
USE_BUILT_BRANCH
or similar with hardcoded value, but that's another topic.
inpsyde/workflows
would be a straight-forward package name :)contains(github.ref, 'refs/tags/')
is really github.ref_type
💡composer compile-assets
npm install
is an update command, issue npm ci
cs2pr
is unused./node_modules/.bin/stylelint
is really npx stylelint
./vendor/bin/phpunit
is really composer exec phpunit
All the best to you! ❤️
-
Not really.
Use npm ci
/ yarn --frozen-lockfile
(yarn install --prefer-offline --frozen-lockfile
as an option) for dependencies installation for workflows where asset compilation is required.
It speeds up dependencies installation and it's recommended way for CI environment.a
Do nothing.
This is a kind of breaking change because we must have yarn.lock
or package-json.lock
in the repository.
But in fact that breaking change done already because actions/setup-node@v3
cache functionality requires lock file to be present.
So all we need to make this clear with documentation.
The single workflow where the "old" cache is used is
reusable-workflows/.github/workflows/build-assets-compilation.yml
Lines 98 to 104 in 3020c4d
The "Set up node" and "Set up PHP" steps are (or should be) identical across all affected workflows. Keeping them aligned when things change is challenging because you need to be aware of which workflows are affected.
Instead of hardcoding the basic node and PHP/Composer setup steps in all workflows, there should be two workflows files that are called in the other workflows (nested reusable workflows; documentation).
How inputs and secrets can be passed to the nested workflow needs to be investigated.
Keep things as-is, and use a matrix sheet for a quick overview of which inputs and secrets are used in which workflows. From this, the state of alignment of the setup steps can then be derived.
No response
Currently the coding standards check can happen only with one PHP version, this is a problem when you want to use this reusable workflow and run the coding standards under several PHP versions.
I am referring to: https://github.com/inpsyde/reusable-workflows/blob/main/.github/workflows/coding-standards-php.yml#L44
It would be great to add the strategy matrix to have this working:
See:
jobs:
static-code-analysis-backend:
name: Static code analysis PHP
runs-on: ubuntu-latest
strategy:
matrix:
php-versions: [ "7.4","8.0","8.1" ]
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-versions }}
tools: composer:v2,cs2pr
- name: Packagist.com Auth
run: composer config --global --auth http-basic.repo.packagist.com token ${{ secrets.PACKAGIST_TOKEN }}
- name: Validate composer.json and composer.lock
run: composer validate
- name: Install composer dependencies (use cache)
uses: ramsey/composer-install@v1
with:
composer-options: "--prefer-dist"
- name: Check coding standards PHP
run: vendor/bin/phpcs --report-full --report-checkstyle=./phpcs-report.xml
- name: Check Psalm
run: vendor/bin/psalm --output-format=github --php-version=${{ matrix.php-versions }}
- name: Annotate coding standards errors PHP
if: always()
run: cs2pr --graceful-warnings ./phpcs-report.xml
Add the matrix strategy
No response
The wp-scripts-lint.yml
workflow does not hardcode default flag values, so whenever a *LINT_ARGS
input is used, calling workflows must pass the default value, too. This is verbose, resulting in a lot of copying & pasting. Also, it's very error-prone because users just don't know they must add default values after their custom flags.
It would be better if the reusable workflow would hardcode sensible default flag values. These would be appended after the *LINT_ARGS
input.
Judging from this search: https://github.com/search?q=org%3Ainpsyde+OR+org%3AInpsyde-Global-Service-Provider+wp-scripts-lint+ARGS+language%3AYAML&type=code, the most often repeated flags are the following:
MARKDOWNLINT_ARGS
: --ignore node_modules
STYLELINT_ARGS
: --formatter github
Alternatively, we would update the documentation to warn users to always add defaults.
It is unclear what value is used when a flag is both set with an input and with the following hardcoded flag.
Edit:
MARKDOWNLINT_ARGS
--ignore LICENSE.md
because "modern" packages ship an extension-less LICENSE
file--ignore node_modules
might be eventually fixed upstream; see igorshubovych/markdownlint-cli#460I think this is consistent with other workflows, but now that I've used these workflows for a while, I'm seeing that "GitHub Annotation" format works very well on PRs, but very badly on regular commits.
In fact, on regular commits the annotations are printed without an indication of the file/line where the error happened, making it very hard to fix the issues.
This is true for all the tools: PHPCS, Psalm, etc.
The ideal scenario would be using "GitHub Annotation" format when the current event is a PR, and standard console output in the other cases.
Originally posted by @gmazzap in #19 (comment)
Not really. But now if have a TypeScript-based project there is no possibility to verify the type system before asset compiling. This means we can't leverage TypeScript checks during PRs review.
The simplest solution is to run such a command: tsc --noEmit -project ./tsconfig.eslint.json --skipLibCheck
. Sure will be nice to inline errors in the files view. But my quick GithHub search there is no maintained and ready-to-use GitHub action.
Maybe we could check https://github.com/Arhia/action-check-typescript but there are a lot of issues without a respond.
So IMO small dependency-less command is better.
Do nothing.
No response
We should create a new reusable workflow "static-wp-scripts-qa.yml" which contains following:
wp-scripts lint-md-docs docs/**/*.md
wp-scripts lint-js resources/{ts,js}/**/*{js,ts,tsx}
wp-scripts lint-style resources/{css,scss}/**/*.{css,scss}
Following inputs are required:
NPM_REGISTRY_DOMAIN:
description: Domain of the private npm registry.
default: https://npm.pkg.github.com/
required: false
type: string
NODE_VERSION:
description: Node version with which the static code analysis is to be executed.
default: 16
required: false
type: string
ESLINT_ARGS:
description: Set of arguments passed to ESLint.
default: '-o eslint_report.json -f json --ext .js,.jsx,.ts,.tsx ./resources'
required: false
type: string
PACKAGE_MANAGER:
description: Package manager with which the dependencies should be installed (`npm` or `yarn`).
default: 'yarn'
required: false
type: string
NODE_OPTIONS:
description: Space-separated list of command-line Node options.
type: string
default: ''
required: false
Following secrets are required:
NPM_REGISTRY_TOKEN:
description: Authentication for the private npm registry.
required: false
Additionally we should update following reusable-workflows and add a notice (to be defined) to point to the new workflow:
There is no way to configure the PHP version for the assets-compilation workflow.
The ability to configure the PHP version for the build-assets-compilation
workflow just like the coding-standards-php
workflows
To merge the following PR:
No response
In the "Build and push assets" workflow, the DEPS_INSTALL
is of type string
:
reusable-workflows/.github/workflows/build-and-push-assets.yml
Lines 21 to 25 in 10652fd
However, it would make more sense if it would be of type bool
.
Change the type of DEPS_INSTALL
input from string
to bool
.
Additionally, update the description to "Whether or not to install dependencies before compiling."
Do nothing and keep things as is.
The "Lint PHP" workflow already uses the bool
type and the more appropriate description:
reusable-workflows/.github/workflows/lint-php.yml
Lines 21 to 25 in 10652fd
According to a search on GitHub, DEPS_INSTALL
input is never used, so it should be safe to change it.
Right now in build-and-push-assets.yml we have 2 inputs: COMPILE_SCRIPT_PROD
and COMPILE_SCRIPT_DEV
. Those are used on either "push to branch" (DEV) or "create tag" (PROD).
It is not possible to bulid DEV-assets when creating a (pre-)release and it is not possible to build PROD-assets when pushing to a branch.
In best case we should introduce some MODE
-input which can bei either "DEV" or "PROD" and is by default empty. This way we have following scenarios covered:
value | description |
---|---|
'' |
empty value falls back to current behavior: push to branch = DEV, create tag = PROD |
'DEV' |
uses the inputs.COMPILE_SCRIPT_DEV script |
'PROD' |
uses the inputs.COMPILE_SCRIPT_PROD script |
Following open topics exists:
MODE
input.The build-and-push-assets.yml
workflow uses encore prod
and encore dev
as defaults for the COMPILE_SCRIPT_PROD
and COMPILE_SCRIPT_DEV
inputs respectively:
reusable-workflows/.github/workflows/build-and-push-assets.yml
Lines 31 to 40 in af92912
However, with the gradual move from Encore to @wordpress/scripts
(see. e.g., #49), we should no longer default to Encore scripts.
Whenever a caller workflow uses Encore, e.g., by not setting the COMPILE_SCRIPT_PROD
/COMPILE_SCRIPT_DEV
inputs explicitly or by using custom Encore-based commands, we should output a deprecation warning, informing the user to use a different command (@Chrico, which one?).
Instead of throwing a deprecating warning, we could immediately change the COMPILE_SCRIPT_PROD
/COMPILE_SCRIPT_DEV
input defaults. However, this would break BC, requiring instant changes in many caller workflows.
Changing the defaults mid-term will be handled in a different GitHub issue.
The psalm script can infer the php version from the composer.json. Normally we used in our composer.json
...
"require": {
"php": ">=7.4",
...
}
...
The configuration from above will make psalm always infer php7.4 as environment, but that can be described via CLI using the option --php-version
- name: Check Psalm
run: vendor/bin/psalm --output-format=github --php-version=${{ matrix.php-versions }}
psalm running with specific inferred version according with PHP matrix
Add the php-version option to psalm script
No response
Right now we have in many places repeating text, inputs or secrets like COMPOSER_AUTH_TOKEN
, ENV_VARS
, GITHUB_USER
, ...
We should move those "repeating global always available" inputs/secrets into a general documentation ( README.md
) maybe with a matrix (where is what used).
Additionaly we should improve the documentation to not have always "short desc, code block, table, advanced code block". The build-and-push-assets.yml
workflow is actually a good example of having "more" documentation for the workflow itself https://github.com/inpsyde/reusable-workflows/blob/main/docs/build-and-push-assets.md
No response
The packages that are using Build, release notes are overwritten with Compiled assets available in....
lately.
This is potentially due to https://github.com/svenstaro/upload-release-action/blob/master/src/main.ts#L77-L83
They have introduced the overwrite
option but we have it fixed here https://github.com/inpsyde/reusable-workflows/blob/main/.github/workflows/build-assets-compilation.yml#L143
Example repos:
Release notes to not be overwritten. Perhaps, a new config/input that we can say true
or false
for the overwrite
value.
No response
No response
No response
Right now we have test-unit-php.yml
and lint-php.yml
which allow on.workflow_call_inputs.PHP_MATRIX
to be set. This JSON string will be used internally to run the reusable workflow against multiple PHP_VERSIONS.
It seems that since 08/2022 Github supports matrix usage when using reusable-workflows https://docs.github.com/en/actions/using-workflows/reusing-workflows#using-a-matrix-strategy-with-a-reusable-workflow . So instead of adding PHP_MATRIX
as a JSON-string we should convert those to on.workflow_call_inputs.PHP_VERSION
and the matrix has to be defined outside in the calling workflow.
before:
name: Unit tests PHP
on:
pull_request:
jobs:
tests-unit-php:
uses: inpsyde/reusable-workflows/.github/workflows/tests-unit-php.yml@main
secrets:
COMPOSER_AUTH_JSON: ${{ secrets.COMPOSER_AUTH_JSON }}
ENV_VARS: >-
[{"name":"EXAMPLE_USERNAME", "value":"${{ secrets.USERNAME }}"}]
with:
PHP_MATRIX: >-
["8.0", "8.1"]
after
name: Unit tests PHP
on:
pull_request:
jobs:
tests-unit-php:
strategy:
matrix:
php: ["8.0", "8.1"]
uses: inpsyde/reusable-workflows/.github/workflows/tests-unit-php.yml@main
secrets:
COMPOSER_AUTH_JSON: ${{ secrets.COMPOSER_AUTH_JSON }}
ENV_VARS: >-
[{"name":"EXAMPLE_USERNAME", "value":"${{ secrets.USERNAME }}"}]
with:
PHP_VERSION: ${{ matrix.php }}
In the 'Create plugin archive' workflow it would be great to have more flexibility for setting archive and directory names.
I would like to be able to include a build number in the archive filename, optionally - creation time, and maybe some other information. By using ARCHIVE_NAME
we only can change the base for the archive name, and the PLUGIN_VERSION
is always appended to it. In my particular case, this leads to filenames like
payoneer-checkout-2-test-include-build-number.zip
where '2' is a build number and 'test-the-build-number' is a PLUGIN_VERSION
.
Another problem is that ARCHIVE_NAME
is also used as a directory name inside the archive (payoneer-checkout-2
in our example), which also prevents us from using custom archive names. Ideally, I would like to set DIR_NAME
separately, while a fallback to the current behavior is still available.
ARCHIVE_NAME
input to set the full archive name without anything appended to it (except for file extension). Alternatively, introduce a new input FULL_ARCHIVE_NAME
to avoid breaking changes.DIR_NAME
input for the directory name inside the archive with a fallback to ARCHIVE_NAME
to avoid breaking changes.I was looking for the ability to override some steps from the reusable workflow
I'm ready to prepare a PR for this while I have some time. But I want to collect opinions on it first.
Related Jira issue https://inpsyde.atlassian.net/browse/PROD-141.
I would like to be able to do:
ASSETS_TARGET_PATHS: './assets ./my-file.txt'
Currently the workflow is creating directories prior to the build see here.
And then instead of doing git add ${path}
is doing git add -f "${path}/*
We could use something like:
ASSETS_TARGET_PATHS: './assets/ ./my-file.json'
If the path ends in /
then it is a directory else a file.
This is a breaking change but it is more flexible.
I considered the following:
We could change the default
Node version 16 LTS has end of life for security support on 11. Sep 2023 and had active support 18 Oct 2022.
We should consider upgrading our workflows (and packages) to Node 18 LTS - active support 18. Oct 2023 and security support 30. Apr 2025.
We could also wait for Node 20 LTS to appear and use straight away the newest version. But first we need to check how support in libraries and build tools is in general for this version.
Using Current HEAD commit in main.
The automatic release process does not create a stable release anymore on main-built branch.
It seems that the current release.config.js branches definition is causing the problem:
This will not work for having Stable releases on main-built branch.
"branches": [
"main",
"main-built",
"next",
"next-built",
{
"name": "beta",
"prerelease": true
},
{
"name": "beta-built",
"prerelease": true
},
{
"name": "alpha",
"prerelease": true
},
{
"name": "alpha-built",
"prerelease": true
}
],
When we provide custom config it works:
["main-built", {"name": "beta-built", "prerelease": true}, {"name": "alpha-built", "prerelease": true} ]
Trigger the release in a main-built branch.
You will get a prerelease instead of a release in GH.
A stable release is created in GH.
No response
No response
No response
Right now only in the tests-unit-php.yml workflow it is possible to inject ENV_VARS
via secrets
.
We should add the ENV_VARS
as secrets
to all workflows we have right now:
parsing and adding them to the env is done by following step:
- name: Set up custom environment variables
env:
ENV_VARS: ${{ secrets.ENV_VARS }}
if: ${{ env.ENV_VARS }}
uses: actions/github-script@v6
with:
script: |
JSON
.parse(process.env.ENV_VARS)
.forEach(envVar => core.exportVariable(envVar.name, envVar.value));
No response
Current implementation does not support publishing to NPM registry.
Align the implementation so the NPM_TOKEN can be applied and publish feature is enabled
None other really
No response
The current build-assets-compilation.yml
sets a ENV variable ENCORE_ENV
which can be used in assets-compiler.json
. The problem is that this env var is called ENCORE_ENV
, which is too specific to a certain implementation.
To have not any breaking change to our (Inpsyde) current workflow, which makes use of Symfony Webpack Encore, I would suggest following:
ENCORE_ENV
with some note like "deprecated, please use ....".BUILD_ENV
.Also, trigger a notice (how?) that is displayed below the action run when the deprecated ENCORE_ENV
input is used.
We could also open this up by introducing a new inputs
which allows to insert a JSON as string which maps
github.ref
( refs/heads/<branch_name>
, refs/tags/<tag_name>
)and/or
github.ref_type
(branch
or tag
)to an BUILD_ENV
No response
Using latest HEAD commit of the package.
When doing a beta or alpha release we are getting inconsistent version names.
Commits have the suffix "-built" appended to the version name while the tag doesn't.
chore(release): 1.0.0-beta-built.1 [skip ci]
This also happens in the replacements done by the plugin:
Version: 1.0.0-beta-built.1
Package.json is also affected
"version": "1.0.0-beta-built.1"
Trigger the workflow from a beta-built branch.
Tag and version name are aligned.
NA
NA
No response
PHP 8.0 has end of life November 2023.
Link: https://www.php.net/supported-versions.php
We should consider upgrading our workflows (and packages) to PHP 8.1 - active support November 2023 and security support November 2024.
We could also consider directly moving to PHP 8.2 min. But first we would need to check the active WordPress support of PHP 8.1 and 8.2 to ensure that we're good to go.
No response
We marked following workflows as deprecated to replace them with the wp-scripts
-workflow:
We are planning to remove those repositories in near future (~November 2023) completely from the inpsyde/reusable-workflow-repository.
No response
Right now we're using in following reusable workflows PHP 7.4 as default:
PHP 7.4 had EoL in 11/2022.
We should upgrade those workflows to PHP 8.0 or even better to 8.1 as minimum.
Branch | released | active support | security support |
---|---|---|---|
8.0 | 11/2020 | 11/2022 | 11/2023 |
8.1 | 11/2021 | 11/2023 | 11/2024 |
leave it as it is..
No response
Xdebug is enabled by default on Ubuntu GitHub Actions images. However, we don't use it in any of the PHP-related workflows. Since it has a negative impact on PHP performance, it should be disabled.
Add coverage: none
to the "Set up PHP" step as described here: https://github.com/shivammathur/setup-php#disable-coverage:
- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
# […]
coverage: none
It is unclear what the actual performance benefit would be, so the alternative is not to add this.
No response
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.