hasura / smooth-checkout-buildkite-plugin

All the things you need during a Buildkite checkout :butter: :kite:

Smooth Checkout

All the things you need during a Buildkite checkout 🧈 🪁

Usage

Repository-less builds

steps:
  - command: echo "Skips checking out Git project in checkout"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          skip_checkout: true

Checking out repo

steps:
  - command: echo "Checks out repo at given ref"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          repos:
            - config:
              - url: [email protected]:<username>/<reponame>.git
                ref: <ref> # (optional)
                clone_flags: <flags> # (optional) flags to use with `git clone` command

If ref is not provided the values of BUILDKITE_BRANCH and BUILDKITE_COMMIT env vars are used.

Allowed values for ref:

• Branch name
• Git tag
• Commit SHA (40 character long hash)

clone_flags can either be a string or an array of strings.

Shallow clone

A shallow clone can easily be done by passing the depth flag in the clone_flags field. For example:

steps:
  - command: echo "shallow clone repo"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          repos:
            - config:
                - url: "[email protected]:hasura/smooth-checkout-buildkite-plugin"
                  clone_flags: "--depth 1"

Checking out multiple repositories

You can checkout multiple repositories by providing multiple config elements:

steps:
  - command: echo "Checks out multiple git repositories"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          repos:
            - config:
              - url: [email protected]:<username>/<repo_1>.git
            - config:
              - url: https://github.com/<username>/<repo_2>.git
                ref: <ref>

Unlike single repo checkouts, when checking out multiple repos, they will each be checked out in subdirectories of $BUILDKITE_BUILD_CHECKOUT_PATH corresponding to the name of the repository (based on its URL). In the above example, the contents of the working directory would be repo_1/ and repo_2/.

You can also explicitly provide the path to an ssh identity file using the ssh_key_path config field:

steps:
  - command: echo "Checks out multiple git repositories"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          repos:
            - config:
              - url: [email protected]:<username>/<repo_1>.git
                ssh_key_path: .ssh/key_1
            - config:
              - url: [email protected]:<username>/<repo_2>.git
                ref: <ref>
                ssh_key_path: .ssh/key_2

If you are using smooth-secrets to configure ssh keys, you can do the following to easily set the ssh_key_path:

steps:
  - command: echo "Checks out multiple git repositories"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          repos:
            - config:
              - url: [email protected]:<username>/<repo>.git
                ssh_key_path: $$SECRETS_DIR/<key>

where <key> is the value of key field in smooth-secrets config with any / characters replaced by -.

Checking out repo from git mirrors

You can attempt to fetch a git repository from git mirrors and fallback to using the original source repo in case of a failure while checking out from mirrors.

steps:
  - command: echo "Checks out repo from mirror (fall back to github in case of failure)"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          repos:
            - config:
              - url: [email protected]:/path/to/git/mirror
              - url: [email protected]:<username>/<reponame>.git

Setup & Cleanup

Smooth Checkout also supports setting custom directories for your jobs and deleting the checkout directory after the job completes. BUILDKITE_BUILD_CHECKOUT_PATH is set to the directory specified by the build_checkout_path option. For legacy reasons, the environment variable WORKSPACE is also set to the same value, but its usage is deprecated.

steps:
  - command: echo "Checks out repo to custom directory"
    plugins:
      - hasura/smooth-checkout#v4.4.1:
          build_checkout_path: /tmp/${BUILDKITE_COMMIT}
          delete_checkout: true
          repos:
            - config:
              - url: [email protected]:<username>/<reponame>.git

Use custom directory with interpolation

Additionally, if BUILDKITE_PIPELINE_NO_INTERPOLATION is set to true and custom directory is an interpolation of variables (example: depends on BUILDKITE_JOB_ID, BUILDKITE_STEP_ID etc); use interpolate_checkout_path env to set the directory.

steps:
  - command: echo "Checks out repo to custom directory"
    plugins:
      - hasura/smooth-checkout#v4.2.1:
          interpolate_checkout_path: /tmp/${BUILD_CHECKOUT_PATH}/${BUILD_ID}
          delete_checkout: true
          repos:
            - config:
              - url: [email protected]:<username>/<reponame>.git

Contributing

• Fork this repo and clone on your machine:

  git clone https://github.com/<your-username>/smooth-checkout-buildkite-plugin

• Make the required changes
• Run linter

  docker-compose run --rm lint

• Run tests (try to add tests for any new features introduced)

  docker-compose run --rm tests

• Once linter and tests run successfully, open a pull request on this repo

smooth-checkout-buildkite-plugin's Issues

error: pathspec 'TAG_NAME' did not match any file(s) known to git

Facing this error when git repository already exists in build node and the build was triggered due to a git tag push.

remove or make sudo dependency to be optional

Currently when buildkite smooth checkout plugin runs at a place that does not contain sudo we have

Removing directory: /root/buildkite-checkouts/3d59b7a7-e1ac-42e7-93b1-c61e9f5ad05e/9c5d1b97-d24f-4879-af0f-b70f68f13a8e/
/buildkite/plugins/github-com-hasura-smooth-checkout-buildkite-plugin-v3-0-0/hooks/pre-exit: line 11: sudo: command not found

One solution might be: check if sudo exists and use sudo, if not fallback to normal operation.

The plugin smooth-checkout checkout hook exited with status 1

I am having this error from dockers in bulidkite because of the smooth-checkout-plugin. Can you guys fix this or assist me in how to get pass this.

(Checking out repository)

[2022-11-28T20:17:27Z] 0

  | [2022-11-28T20:17:27Z] :::: local clone location = '.'
  | [2022-11-28T20:17:27Z] :::: checkout ref = '88cae23eb4a74df81d2704d28769509e4ca3e93a'
  | [2022-11-28T20:17:27Z] :::: git remote url = '[email protected]:Vydia/web.git'
  | [2022-11-28T20:17:27Z] :::: ssh key path = ''
  | [2022-11-28T20:17:27Z] :::: clone flags = '--depth 1'
  | [2022-11-28T20:17:27Z] Git repository already exists.
  | [2022-11-28T20:17:27Z] So, doing a git clean and setting the remote URL
  | [2022-11-28T20:17:27Z]
  | [2022-11-28T20:17:27Z] We trust you have received the usual lecture from the local System
  | [2022-11-28T20:17:27Z] Administrator. It usually boils down to these three things:
  | [2022-11-28T20:17:27Z]
  | [2022-11-28T20:17:27Z] #1) Respect the privacy of others.
  | [2022-11-28T20:17:27Z] #2) Think before you type.
  | [2022-11-28T20:17:27Z] #3) With great power comes great responsibility.
  | [2022-11-28T20:17:27Z]
  | [2022-11-28T20:22:27Z] [sudo] password for buildkite-agent:
  | [2022-11-28T20:22:27Z] sudo: timed out reading password
  | [2022-11-28T20:22:27Z] 1
  | [2022-11-28T20:22:27Z] 🚨 Error: The plugin smooth-checkout checkout hook exited with status 1

Check out single repository in subdirectory

Unlike single repo checkouts, when checking out multiple repos, they will each be checked out in subdirectories of $BUILDKITE_BUILD_CHECKOUT_PATH corresponding to the name of the repository (based on its URL). In the above example, the contents of the working directory would be repo_1/ and repo_2/.

I see it is supported when checking out multiple repos, but was wondering if the same can be setup for single checkout scenario while keeping the same $BUILDKITE_BUILD_CHECKOUT_PATH

e.g.

want to check out repo to
$BUILDKITE_BUILD_CHECKOUT_PATH/test_repo instead of $BUILDKITE_BUILD_CHECKOUT_PATH

Issues while using the versions v4.2.1 and v3.1.1

I need to use the latest version due to the change in GITHUB.COM RSA keys. Kindly release the new version with the below issues fixed pls.

1. Error while using the version 3.1.1

🧹 Cleaning up workspace | 1m 29s
-- | --
  | Removing directory: /var/lib/buildkite-agent/buildkite-checkouts/018725ce-3365-48a3-bd56-cab71be669c0/018725d2-6c21-4972-bbae-5c137129b7cd/
  |  
  | We trust you have received the usual lecture from the local System
  | Administrator. It usually boils down to these three things:
  |  
  | #1) Respect the privacy of others.
  | #2) Think before you type.
  | #3) With great power comes great responsibility.
  |  
  | [sudo] password for buildkite-agent: # Received cancellation signal, interrupting
  |  
  | 🚨 Error: Error tearing down bootstrap: The plugin smooth-checkout pre-exit hook exited with status -1

2. Error while using the version 4.2.1

_📂 Setting up workspace | 0s
-- | --
  | Creating directory: /var/lib/buildkite-agent/builds/buildkite-tellus-microservice-1-i-001cff39f0882222a-1/organic-rssssse-1/tellus-microservice
  | Setup completed successfully.
  | 🔑 Setting up git and ssh | 0s
  | Checking out repository |  
  | 0
  | :::: local clone location = '.'
  | :::: checkout ref = 'master'
  | :::: git remote url = '[email protected]:orgaxxxxxxxx/portal-dddd.git'
  | :::: ssh key path = ''
  | :::: clone flags = ''
  | Git repository already exists.
  | So, doing a git clean and setting the remote URL
  |  
  | We trust you have received the usual lecture from the local System
  | Administrator. It usually boils down to these three things:
  |  
  | #1) Respect the privacy of others.
  | #2) Think before you type.
  | #3) With great power comes great responsibility.
  |  
  | [sudo] passw_

repo not checked out with latest changes when using `ref` option

In cases when we do a git clean, and checkout a ref that is already present locally, the latest changes are not checked out. git fetch just fetches the new changes if any but doesn't update the local ref so git checkout checks out old version of the code. For eg., let's say we had checked out a branch with commit-a in a job X. the job exits and we push a new commit commit-b to the branch. the job is picked up by the same agent. since the repo already exists locally, we do a git clean and then git fetch the branch. git fetch fetches the new commit but doesn't update the lcoal branch ref. we checkout the branch with git checkout but it's checked out at commit-a not commit-b

Eg. https://buildkite.com/hasura/deploy-hasura-cluster-mono/builds/38#01893e54-bb48-441f-b047-151bd5ef2064
The branch is on ref 3367fa41c313, but the repo is checked out at 13048e6713

Document about using the plugin without any arguments

When the plugin is used without any arguments, it seem to automatically detect values from BUILDKITE_REPO, etc. We should probably add this use case to the README

Enable shellcheck linting in CI

Modify https://github.com/hasura/smooth-checkout-buildkite-plugin/blob/main/.github/workflows/lint.yml to install and run shell check

Inconsistent hook behavior when checking out multiple repos

I have a pipeline step that is checking out 2 separate repos, each that have a .buildkite/hooks/pre-command file. The first repo it is checking out has its own pipelines and hooks, and the second repo I am checking out is the repo where this pipeline being executed lives.

When this step gets run, sometimes it loads the pre-command hook and sometimes it does not. When the step fails, I can see in the log that it did not attempt to run the "local pre-command hook", only the "global pre-command hook". If I retry that step multiple times, eventually it runs the local pre-command hook and then my step succeeds.

Please let me know if I am misunderstanding how hooks should function with the multi-repo checkout. Ideally the hooks from each repo would be executed with the option to opt out.

Support default checkout directory

Currently, all the checkouts happen in $HOME/builds/<HOSTNAME>/<PIPELINE_SLUG>

This will be problematic when trying to run builds parallelly on the same machine.

So for every build, we will first create a unique directory where our script will operate. (mkdir -p and cd into this directory at the top of our checkout script)

This unique directory will be of the format $HOME/buildkite-checkouts/$BUILDKITE_BUILD_ID/$BUILDKITE_JOB_ID (check if the envs mentioned in the format are accessibly correctly in the script. If not we could modify the format

We should also clean up the checkout directory in the pre-exit hook.

set `BUILDKITE_BUILD_CHECKOUT_PATH` env var

We make use of WORKSPACE env var which is set in the environment hook, but it would be good to support the buildkite's built-in env var BUILDKITE_BUILD_CHECKOUT_PATH as well

Can’t checkout plugin without a plugins-path

Hi,

I have added the steps,

steps:
  - command: echo "Check out signing repo"
    plugins:
      - hasura/smooth-checkout#v3.1.0:
          repos:
            - config:
                - url: [email protected]:Digital/ios-match-certificates.git

however, i get “:rotating_light: Error: Failed to checkout plugin smooth-checkout: Can’t checkout plugin without a plugins-path”

Do I need to add the plugins-path after "hasura/smooth-checkout#v3.1.0:"

redirect clean up logs to /dev/null

Currently we remove the checkout files during the cleanup stage here

smooth-checkout-buildkite-plugin/hooks/pre-exit

Lines 12 to 13 in de4f6b9

	rm -rf "$BUILDKITE_BUILD_CHECKOUT_PATH" \
	|| sudo rm -rf "$BUILDKITE_BUILD_CHECKOUT_PATH"

If there are lot of files and if the clean requires a sudo then we see a lot of permission denied log lines in the cleanup stage which is polluting the actual job logs. Hence it would be best to avoid logging those lines.

Feature: Support configurable number of retries

Would you be willing to accept a PR that adds a retry option?

Github.com had a tiny outage today, and so some builds failed because they couldn't pull. It's not a big deal but I thought that if there was a retry mechanism it could have been avoided.

Support Checking out code

We will introduce configurations needed for checking out the code for a ref

A ref could be

• a branch name
• a commit sha
• a git tag

steps:
  - command: echo "Skips checking out Git project in checkout" 
    plugins:
      - hasura/smooth-checkout#v1.0.0:
          clone_url: $BUILDKITE_REPO
          ref: $BUILDKITE_TAG

Use default checkout path

I'm interested in using this plugin to check out a different branch than the default (which seems like something Buildkite itself should make configurable, but no matter). I don't want the behavior of creating new checkout directories per build though. That seems like unnecessary overhead and I don't think the explanation of it is correct. Buildkite uses the agent id, not the hostname, for the directory name, so if you've got multiple agents on the same host, they will use different directories (in fact this is a bit annoying sometimes because if you change the agent name you end up with orphaned directories using up disk space). Here's an example build that's running on a VM that has 4 agents configured with spawn: https://buildkite.com/llvm-project/upstream-bazel-rbe/builds/5658. Note that the working directory is the same as the agent name (I can tell you that is not the hostname). If you want to configure the build path per-agent you can also set that with https://buildkite.com/docs/agent/v3/configuration#build-path (note that will not be dynamic per job).

So I'd like to either 1) remove the custom checkout directory entirely, or if you still have some use for it 2) make it optional. Happy to send a PR

error while cloning multiple private repositories

https://buildkite.com/hasura/graphql-engine-mono/builds/2166#df686b8c-09d3-4583-a7e9-0d874be18587

SSH keys are added using the smooth-secrets plugin. When cloning through git@github:<org>/<repo> urls, the first key that matches the host(i.e. github.com) is accepted for authentication. But, since that key doesn't have permissions to clone other private repositories, the git clone fails.

Github suggests adding aliases in ssh config. But this seems hard to implement because we will need to modify the smooth-secrets plugin to add hostname aliases for the ssh keys that it adds and then use those aliases here. It would make this plugin dependent on the smooth-secrets plugin.

I propose that we add an optional config field key_path with which we can specify the path to the private key and use this approach to clone the repo(this method has already been used in our system here)

support cloning multiple git repositories

We should probably allow something like having two smooth-checkout section or perhaps somehow give the config in a single plugin call to provide multiple repositories for cloning (note: in case of the second case we will need to think about mirrors implementation we already have and not break it)

steps:
  - command: echo "Checks out multiple repositories"
    plugins:
      - hasura/smooth-checkout#v2.0.0:
          config:
            - url: [email protected]:<username>/<reponame1>.git
      - hasura/smooth-checkout#v2.0.0:
          config:
            - url: [email protected]:<username>/<reponame2>.git

make configurable base checkout path

Currently we use $HOME/buildkite-checkouts as the base path for WORKSPACE

smooth-checkout-buildkite-plugin/hooks/environment

Line 13 in 764af8a

set_env WORKSPACE "$HOME/buildkite-checkouts/$BUILDKITE_BUILD_ID/$BUILDKITE_JOB_ID"

But there are cases where we might be interested in using a different path such as /buildkite/builds instead of $HOME/buildkite-checkouts