jonico / pscale-workflow-helper-scripts

Workflows and helper scripts around the PlanetScale DB workflow to automate database branch creation, association, update and merge directly out of your pull/merge request or favourite CI/CD. This is just a passion project, no official repository from Planetscale.

PlanetScale Workflow Helper scripts

📺 DevOpsPro Talk

📺 Recording from live demo at PlanetScale Happy-Half-Hour

📺Longer talk at WeAreDevelopers DevOps Day

This is a collection of GitHub Workflows and CI/CD helper scripts around PlanetScale's database branching workflows to:

• attach, create, update, approve, merge and delete PlanetScale branches and deploy requests directly from your Pull Requests
• wait for branch creation to be completed
• wait for deploy requests to be successfully merged (and get an update on the current migration status)
• DB connection string creation/parsing and extending scope of service tokens
• examples on how to create branches and deploy requests (branches, credentials, deploy requests) in any CI/CD environment
• creating one-time links with connection info for feature branches that correspond to your preview environments
• run pscale from a Docker container and still use your local config files, service tokens and port forwarding
• delete databases

IssueOps Example

(example PR)

Installation

To test out the IssueOps commands and helper scripts, either work on a copy of this repo or run the following command in the root directory of your GitHub repo's workspace and follow the suggestions of the script:

curl https://raw.githubusercontent.com/jonico/pscale-workflow-helper-scripts/main/install.sh | bash

Issue-Ops based workflows

(GitHub Actions workflow source)

/ps-create: Create PlanetScale Deploy Request from your PR

You can create a database branch and deploy request in an already existing Pull Request using /ps-create:

See configuration section for an explanation of the environment variables to set to specify a PlanetScale org and database different from the default settings.

/ps-attach: Attach existing PlanetScale branch or Deploy Request to your PR

If the PlanetScale deploy request or database branch and your pull request both exist already, you can use the /ps-attach command, followed by the PlanetScale URL to your branch/DR:

/ps-update: Refresh schema info in your PR from your attached Deploy Request

If you like to refresh the schema information of your branch or add additional DDL statements, use /ps-update:

/ps-approve: Approve Deploy Request from your PR

Approvals can be done using /ps-approve:

/ps-merge: Merge Deploy Request attached to your PR

Schema changes in attached deploy requests can then be applied to production by adding a /ps-merge comment to the pull request. This IssueOps command will also create GitHub deployment events, so that you could have other integrations and Action based workflows hook into the deployment lifecycle.

/ps-delete: Delete PlanetScale branch attached to your PR

Finally, you can delete the database branch associated with a Git branch using /ps-delete:

Stay tuned for updates and any feedbacks / PRs are welcome 😊

Zero-setup Action workflows included ✨

In case you like to see those helper scripts in action without any further manual configuration, just

1. Create a copy of this repo by clicking on the green 'Use this template' button or just here. Both private and public repo visibility work.

2. Create a PlanetScale account: If you have not done already, sign up here for a PlanetScale account, you can have one database for free. Do not create a database yet though, this step will be automated as part of the first Action workflow.

3. Switch to the Actions tab in your repo

4. Start the workflows manually, in the displayed order.

See configuration section for an explanation of the environment variables to set to specify a PlanetScale org and database different from the default settings.

5. When you run the workflows, follow the output where it will ask you to authenticate to PlanetScale by clicking the displayed links.

If you start the "Create branch/PR for schema change" workflow, a pull request like this one will be created that is associated with a PlanetScale deploy request and the desired schema changes.

If you like to install the workflows in your own repository, follow the installation steps.

Configuration

All scripts and IssueOps command in this repo have been designed to run successfully even if zero configuration was provided. If the defaut values do not fit your you, or you do not like to always authorize every action with the Web UI, here are the environment variables you can override in your CI/CD or set as GitHub secrets for your GitHub Action workflows to override default behavior.

DB_NAME

DB_NAME is name of your PlanetScale database. Unless you attach a specific branch or deploy request to your PR, it defaults to example-db-<yourgithubaccount>. If you rather like to change the source for the default DB name, it would be this file.

ORG_NAME

ORG_NAME is name of your PlanetScale organization. Unless you attach a specific branch or deploy request to your PR, it defaults to the first PlanetScale org you joined with your account. If you rather like to change the source for the default DB name, it would be this file.

PLANETSCALE_SERVICE_TOKEN_ID and PLANETSCALE_SERVICE_TOKEN

If you do not like to authorize actions by clicking on authorization links in the browser, you can create a PlanetScale service token. Don't forget to set the proper scopes for the database operations you like to automate.

Note: While service tokens can be used to work with existing databases, there is currently no scope to create new databases. This is why the scripts / workflows to create entirely new databses will always prompt you to authenticate via the Web UI.

CI - Runnning in CI

GitHub Actions and many other CI/CD platforms set the CI variable to true when they run a script. The helper scripts in the .pscale/cli-helper-scripts/ directory will only prompt for interactive user input if the CI variable is not set. In CI mode, it will output additional workflow commands to standard out to enable communication between GitHub Action workflow steps. If you are using a different CI/CD platform than GitHub Actions, you can ignore all lines starting with "::".

Initial table

When you create a bew database from scratch, a default table called pixel_matrix will be created. If you like to change this behavior, modify this file.

pscale-workflow-helper-scripts's Issues

pscale helper itself

URL to usage example (e.g PR or repo URL)

#2

Link to tweet mentioning your example and @planetscaledata

https://twitter.com/planetscaledata/status/1468120594063396868

Like to get contacted in case you won some swag?

• Interested in swag

Slack account in Vitess slack channel

Johannes Nicolai

Error: both --service-token and --service-token-id are required for service token authentication

Basically its not possible to use pscale CLI within github action. In this example im using create-deploy-request function in ps-create-helper-functions.sh

As you can see below, the user gets authenticated, however requests for authentication anyways

I`m able to perform the same action locally

There is any new business rule that doest not allow to use CLI in CI/CD?

Job:

jobs:
  deploy-planetscale:
    name: Planetscale deploy
    runs-on: ubuntu-latest
    #container: node:latest
    steps:
      - name: checkout
        uses: actions/checkout@v2

      - name: Apply database migrations in pre-deploy database branch
        timeout-minutes: 3
      # - shell: bash
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          PLANETSCALE_SERVICE_TOKEN_ID: ${{secrets.PLANETSCALE_SERVICE_TOKEN_ID}}
          PLANETSCALE_SERVICE_TOKEN_NAME: ${{secrets.PLANETSCALE_SERVICE_TOKEN_NAME}}
          PLANETSCALE_SERVICE_TOKEN: ${{secrets.PLANETSCALE_SERVICE_TOKEN}}
          ORG_NAME: ${{secrets.ORG_NAME}}
          DB_NAME: ${{secrets.DB_NAME}}
          BRANCH_NAME: pre-deploy

        run: |
          ./scripts/deploy-database-staging.sh

deploy-database-staging.sh

#!/bin/bash
. ./scripts/planetscale-docker-image.sh

unset PLANETSCALE_SERVICE_TOKEN
. ./scripts/authenticate-planetscale.sh

#npm i -g prisma >3
#prisma db push --skip-generate

. ./scripts/planetscale-helper-functions.sh
create-deploy-request "$DB_NAME" "$BRANCH_NAME" "$ORG_NAME"
create-deployment "$DB_NAME" "$ORG_NAME" "$DEPLOY_REQUEST_NUMBER"

Obs: All env vars are set propelly

issue running export-db-connection-string.sh and accessing the output

Good Morning,

I am trying to run export-db-connection-string.sh (https://github.com/jonico/pscale-workflow-helper-scripts/blob/main/.pscale/cli-helper-scripts/export-db-connection-string.sh) within my yml file:

this is the error I get in GitHub:

furthermore, after I have successfully managed to run the script, I would need to store the output (connection string) as a global variable, so that i can use it in the next job (docker build), what is the correct way to access "set-output name=dbconnection_${i}_${j}" within my yml file

Kind Regards
AAACafe

Branch diff is always null

I'm able to successfully attach and update a PR with a Planetscale DR using the issue-ops-ps-commands /ps-attach and /ps-update, however the Branch diff is always null.

Even thought the workflow job passes (green), I also get the following error {"error": "deploy request 'myapp/7' does not exist in organization my-org"}. This DR does indeed exist, and has schema changes, so I'm unsure where the issue stems from. Note that I am only using the issue-ops-ps-commands.yml workflow along with the directory of .pscale shell commands.

Prisma Example

This is fantastic. Thanks for putting this out there! Super helpful. Are you planning on adding an example demoing something like this with Prisma in CI/CD pipeline? If not, I could maybe throw something together and submit a PR.