xenitab / azure-devops-templates Goto Github PK

Working in monorepos we sometimes make changes to multiple services at the same time (backend and frontend as an example). This triggers multiple CD runs at approximately the same time and then we get errors in git because we're trying to push when we're not at the latest master.

The fix is to do a git pull --rebase just before we push so that we're on the latest at the time of pushing. We could also catch any error while pushing and try to do this only then.

Get the following error when running the sync cronjob for cloning github actions.
This is due to low access.
Need to give the account access to workflows we also need to document what needs to be configured in this repo.

Already on 'main'
Your branch is up to date with 'origin/main'.
From https://github.com/XenitAB/azure-devops-templates

• branch main -> FETCH_HEAD
• [new branch] main -> upstream/main
  Updating 942334e..1f2adfb
  Fast-forward
  .github/workflows/terraform-docker.yaml | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
  To https://github.com/organiaztion1/azure-devops-templates
  ! [remote rejected] main -> main (refusing to allow a GitHub App to create or update workflow .github/workflows/terraform-docker.yaml without workflows permission)
  error: failed to push some refs to 'https://github.com/organiaztion1/azure-devops-templates'

We need to be able to change the opaBlastRadius for terraform pipelines using variables in the AzDO gui.

When commiting to master (non-tagged), all stages will "run" (be green) but only a dev will run the jobs and the others (qa/prod) will be skipped.

When you add an approval to the prod environment, it will always stop there even if not a tagged release and you need to approve it before it can be "skipped".

Create variables for binaries (version) and their SHA. Also validate SHA before using them.

Currently anyone using self-hosted runners have to manually import the repository to there organization to make it possible to run github workflows.
I have tried to explain what I have done: https://github.community/t/workflow-usage-between-organizations-forks/210558
and hopefully we will get some feedback on this and a new feature or two out of github.

Add documentation for permissions in Azure DevOps

Update the terraform-docker Makefile so we have support for windows https://github.com/XenitAB/azure-devops-templates/blob/main/terraform-docker/README.md

Probably easiest to have seperate commands.
Write something about it in the README

Don't pull when running verify image during push stage. Should be possible with az cli.

Could something like gitleaks be useful as part of our default CI pipeline for checking for passwords being commited in git.
There are probably other tools but gitleaks looks a solid option at least.

https://github.com/zricethezav/gitleaks
https://github.com/zricethezav/gitleaks-action

Should it be twice? Seems to be working.

Due to how github workflow works we have to import this repository to other github organizations if we want to use self-hosted runners.
For more explanation see: #98

While the features get's created we need a way to automatically update the azure-devops-templates imported from other organizations just like we do for azure-devops.

In a pipeline I got the below error.

  ------------------------------------------
  disk i/o             8.274747ms
  parsing HCL          54µs
  evaluating values    4.889227ms
  running checks       2.784716ms

  counts
  ------------------------------------------
  files loaded         11
  blocks               92
  evaluated blocks     92
  modules              0
  module blocks        0

No problems detected!

run make validate and commit changes
##[error]Script failed with error: Error: The process '/bin/bash' failed with exit code 1

This is due to the user removed .terraform.lock.hcl by mistake so when the validation pipeline run:s and we then check if anything have changed we can't see any error because of we don't visualize none index files.
To make the error more visible we need to update in:

In our terraform pipelines in github we currently have a warning telling us to update a few config values.
Look at the blog below to find out how to do it.

https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Currently when a status check fails it prints a very short message.

Commit status is failed

This gives very little information of what is going on and should be expanded with more information.

Maybe we can use the following tool to lint Dockerfiles: https://github.com/hadolint/hadolint

Should perhaps be optional? Default should maybe be off or on but not exit 1 if failing for backward compatibility?

The sync pipeline should also sync branches for development work:
https://github.com/XenitAB/azure-devops-templates/blob/master/.ci/pipeline.yaml

When using this for a monorepo it's not unlikely there will be commits to other services between a push and a tag, this then breaks when we check for the commit hash since we're using Build.SourceVersion which just checks the latest hash that triggered the build.

We need a different way of finding the hash, either by looking at previous pipeline runs for the service or by looking at the dev yaml.

Add ability to scan image using trivy after build.

Copy logic for Dockerfile linting.

How should PR validation / PR triggers be handled?

I can't see this error in our XKS pipelines but for one tenant I can see it.
When I run make locally on my computer it works without any issues.

Digest: sha256:5ee2e1d098e1228e7c49387b9cf570f567b662237ba289a11b7d926fcc1d1849
Status: Downloaded newer image for ghcr.io/xenitab/github-actions/tools:2022.10.1
[Errno 13] Permission denied: '/work/.azure/versionCheck.json'
Traceback (most recent call last):
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/cloud.py", line 474, in get_active_cloud_name
    return cli_ctx.config.get('cloud', 'name')
  File "/opt/az/lib/python3.10/site-packages/knack/config.py", line 99, in get
    raise last_ex  # pylint:disable=raising-bad-type
  File "/opt/az/lib/python3.10/site-packages/knack/config.py", line 94, in get
    return config.get(section, option)
  File "/opt/az/lib/python3.10/site-packages/knack/config.py", line 208, in get
    return self.config_parser.get(section, option)
  File "/opt/az/lib/python3.10/configparser.py", line 782, in get
    d = self._unify_values(section, vars)
  File "/opt/az/lib/python3.10/configparser.py", line 1153, in _unify_values
    raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'cloud'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/az/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/az/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/__main__.py", line 38, in <module>
    az_cli = get_default_cli()
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/__init__.py", line 910, in get_default_cli
    return AzCli(cli_name='az',
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/__init__.py", line 86, in __init__
    self.cloud = get_active_cloud(self)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/cloud.py", line 555, in get_active_cloud
    return get_cloud(cli_ctx, get_active_cloud_name(cli_ctx))
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/cloud.py", line 477, in get_active_cloud_name
    _set_active_cloud(cli_ctx, default_cloud_name)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/cloud.py", line 468, in _set_active_cloud
    cli_ctx.config.set_value('cloud', 'name', cloud_name)
  File "/opt/az/lib/python3.10/site-packages/knack/config.py", line 161, in set_value

Could the following be something to look in to?

https://learn.microsoft.com/en-us/cli/azure/azure-cli-configuration?view=azure-cli-latest#cli-configuration-file

environment {
    AZURE_CONFIG_DIR = "${env.WORKSPACE}/.azure"
}

We are working on supporting AWS in XKF and as a part of that we should add support for AWS in gitopsv2 templates.

Imrpove the docs around: https://github.com/XenitAB/azure-devops-templates/tree/main/gitops-v2

For example give a quick overview of what new.yaml actually do.

Add condition not to run Image scan / Dockerfile lint when tagged

Add template pipeline for packer

Reproduce using the following steps:

• Commit to master for service
• CI / CD runs fine
• Tag service
• CI / CD runs fine
• Update gitops repo (change CD reference tag for example)
• Manually run CD
• Push validation will go through
• Image tag won't be changed
• flux-status-cli will check for status of the latest gitops commit which will never be updated by flux-status

The pool being used by the sync job is deprecated (or at least has stopped working):

For smoe reason when importing this repo to azure devops and run the pipeline I'm getting the following error.
I ran the pipeline multiple times and still the same issue.
It's probably something simple but I don't understand why.

I have also recreated this in a known working environment with a new repo and I get the same issue.
But the pipeline work in old repos.

We need to add scans (tivry, horusec, hadolint) for the GitHub actions

https://docs.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash

Today it's up to the developers to write correct yaml out of the box and the code reviewer have to see that it's correct or clone down the repo and build the yaml on there own.

Implement a simple kustomize build check to see that can at least be built.
It would also be nice to have rego to check for some best practices.
We could also do some general linting of the yaml files.

and probably many more solutions but this is a start.

For some reason the push image takes allot of time when running github actions.

This should only take a few seconds to download and push the image but instead it almost takes a minute.
It's possible that we get a new server between the build and the push but even so it takes a long time.
Compare it to how azure devops times and do some testing around it and see if we can come up with why it takes so long time.

The size in this example is about 141 MB

This example the size is 104 Mb.

It might be that github:s artifacts storage is just slow...

https://github.com/XenitAB/azure-devops-templates/tree/main/gitops-v2-github

https://github.com/XenitAB/azure-devops-templates/blob/main/.github/workflows/push-image-acr.yaml

Create a new GitOps workflow to use PRs instead of environments.

https://buildsec.github.io/frsca/

How resonable would it be for us to make a azure devops implementation of above?
It would be an excellent feature for XKF to be able to provide SLSA compliant CI pipeline out of the box.

Or do we think changing our CI pipeline over to tekton is a good solution?

I created a upstream issue about this: buildsec/frsca#339

Evaluate if it's possible to run static analysis of the code.

Test: https://github.com/ZupIT/horusec

Once #89 is merged, this repo needs a new name. I propose "devops-pipelines".

Add ability to inject steps/jobs/stages to templates in an easy way.

Example: inject step before Docker build to authenticate to maven.

When there are multiple repositories all using these templates, they will all reference the version used individually. This will result in a lot of manual work when updating the version. We need to publish a best practice guide for how to avoid this work.