google-github-actions / setup-gcloud

A GitHub Action for installing and configuring the gcloud CLI.

setup-gcloud GitHub Action

Configures the Google Cloud SDK in the GitHub Actions environment. The Google Cloud SDK includes both the gcloud and gsutil binaries.

Or integrate natively with other Google Cloud GitHub Actions:

• Authenticate to Google Cloud
• Deploy a Cloud Run service
• Deploy an App Engine app
• Deploy a Cloud Function
• Access Secret Manager secrets
• Upload to Cloud Storage
• Configure GKE credentials

This is not an officially supported Google product, and it is not covered by a Google Cloud support contract. To report bugs or request features in a Google Cloud product, please contact Google Cloud support.

Prerequisites

• This action requires Google Cloud credentials to execute gcloud commands. See Authorization for more details.

• This action runs using Node 20. If you are using self-hosted GitHub Actions runners, you must use a runner version that supports this version or newer.

Usage

jobs:
  job_id:
    # Add "id-token" with the intended permissions.
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
    - id: 'auth'
      uses: 'google-github-actions/auth@v2'
      with:
        workload_identity_provider: 'projects/123456789/locations/global/workloadIdentityPools/my-pool/providers/my-provider'
        service_account: '[email protected]'

    - name: 'Set up Cloud SDK'
      uses: 'google-github-actions/setup-gcloud@v2'
      with:
        version: '>= 363.0.0'

    - name: 'Use gcloud CLI'
      run: 'gcloud info'

Inputs

Cloud SDK inputs

• skip_install: (Optional) Skip the gcloud installation and use the system-installed gcloud instead. This can dramatically improve workflow speeds at the expense of a slightly older gcloud version. Setting this to true ignores any value for the version input. If you skip installation, you will be unable to install components because the system-install gcloud is locked. The default value is false. ⚠️ Be aware that GitHub plans to remove the system-installed gcloud, and any workflows with skip_install: true will stop working when that happens.

• version: (Optional) A string representing the version or version constraint of the Cloud SDK (gcloud) to install (e.g. "290.0.1" or ">= 197.0.1"). The default value is "latest", which will always download and install the latest available Cloud SDK version.

  - uses: 'google-github-actions/setup-gcloud@v2'
    with:
      version: '>= 416.0.0'

  If there is no installed gcloud version that matches the given constraint, this GitHub Action will download and install the latest available version that still matches the constraint.

  Warning! Workload Identity Federation requires version 363.0.0 or newer. If you need support for Workload Identity Federation, specify your version constraint as such:

  - uses: 'google-github-actions/setup-gcloud@v2'
    with:
      version: '>= 363.0.0'

  You are responsible for ensuring the gcloud version matches the features and components required. See the gcloud release notes for a full list of versions.

• project_id: (Optional) Project ID (not project number) of the Google Cloud project. If provided, this will configure the gcloud CLI to use that project ID for commands. Individual commands can still override the project with the --project flag. If unspecified, the action attempts to find the "best" project ID by looking at other inputs and environment variables.

• install_components: (Optional) List of Cloud SDK components to install specified as a comma-separated list of strings:

  install_components: 'alpha,cloud-datastore-emulator'

Authorization

The setup-gcloud action installs the Cloud SDK (gcloud). To configure its authentication to Google Cloud, you must first use the google-github-actions/auth action. The auth action sets Application Default Credentials, then the setup-gcloud action references these credentials to configure gcloud credentials . You can authenticate via the following options:

Workload Identity Federation (preferred)

⚠️ You must use the Cloud SDK version 390.0.0 or later to authenticate the bq and gsutil tools.

jobs:
  job_id:
    # Add "id-token" with the intended permissions.
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
    - id: 'auth'
      uses: 'google-github-actions/auth@v2'
      with:
        workload_identity_provider: 'projects/123456789/locations/global/workloadIdentityPools/my-pool/providers/my-provider'
        service_account: '[email protected]'

    - name: 'Set up Cloud SDK'
      uses: 'google-github-actions/setup-gcloud@v2'

    - name: 'Use gcloud CLI'
      run: 'gcloud info'

Service Account Key JSON

jobs:
  job_id:
    steps:
    - id: 'auth'
      uses: 'google-github-actions/auth@v2'
      with:
        credentials_json: '${{ secrets.GCP_CREDENTIALS }}'

    - name: 'Set up Cloud SDK'
      uses: 'google-github-actions/setup-gcloud@v2'

    - name: 'Use gcloud CLI'
      run: 'gcloud info'

Self-hosted runners on Google Cloud Platform

If you are using self-hosted runners that are hosted on Google Cloud Platform, credentials are automatically obtained from the service account attached to the runner. In this scenario, you do not need to run the google-github-actions/auth action.

jobs:
  job_id:
    steps:
    - name: 'Set up Cloud SDK'
      uses: 'google-github-actions/setup-gcloud@v2'

    - name: 'Use gcloud CLI'
      run: 'gcloud info'

Multiple Service Accounts

To use multiple service accounts, a second auth step is required to update the credentials before using setup-gcloud:

jobs:
  job_id:
    # Add "id-token" with the intended permissions.
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
      - id: 'auth service account 1'
        uses: 'google-github-actions/auth@v2'
        with:
          workload_identity_provider: 'projects/123456789/locations/global/workloadIdentityPools/my-pool/providers/my-provider'
          service_account: '[email protected]'

      - name: 'Set up Cloud SDK'
        uses: 'google-github-actions/setup-gcloud@v2'

      - name: 'Use gcloud CLI'
        run: 'gcloud auth list --filter=status:ACTIVE --format="value(account)"'
        # [email protected]

      - id: 'auth service account 2'
        uses: 'google-github-actions/auth@v2'
        with:
          credentials_json: '${{ secrets.GCP_CREDENTIALS }}'

      - name: 'Set up Cloud SDK'
        uses: 'google-github-actions/setup-gcloud@v2'

      - name: 'Use gcloud CLI'
        run: 'gcloud auth list --filter=status:ACTIVE --format="value(account)"'
        # [email protected]

Versioning

We recommend pinning to the latest available major version:

- uses: 'google-github-actions/setup-gcloud@v2'

While this action attempts to follow semantic versioning, but we're ultimately human and sometimes make mistakes. To prevent accidental breaking changes, you can also pin to a specific version:

- uses: 'google-github-actions/[email protected]'

However, you will not get automatic security updates or new features without explicitly updating your version number. Note that we only publish MAJOR and MAJOR.MINOR.PATCH versions. There is not a floating alias for MAJOR.MINOR.

setup-gcloud's Issues

UUID file left behind after run with export_default_credentials=true

With export_default_credentials=true, after the setup-gcloud Github Action is run, a file is left behind in the root of the repository. The filename looks like just a GUID. This leaves the repository in an unclean state that can mess with some builds.

Seems that this is a file used to persist credentials to downstream steps
https://github.com/GoogleCloudPlatform/github-actions/blob/5f5ef3df93d08dce64f649240ddd69b89cab7f19/setup-gcloud/src/setup-gcloud.ts#L88

I don't know if this is avoidable, but since this is pretty surprising behavior it would be nice if it was documented

Can not auth

Question

Can not auth

Run GoogleCloudPlatform/github-actions/setup-gcloud@master

Successfully installed gcloud Cloud SDK
/opt/hostedtoolcache/gcloud/270.0.0/x64/bin/gcloud --quiet auth activate-service-account *** --key-file /tmp/tmp-27025Sjs2nWuhTxI
WARNING: .p12 service account keys are not recomended unless it is necessary for backwards compatability. Please switch to a newer .json service account key for this account.
ERROR: (gcloud.auth.activate-service-account) PyOpenSSL is not available. If you have already installed PyOpenSSL, you will need to enable site packages by setting the environment variable CLOUDSDK_PYTHON_SITEPACKAGES to 1. If that does not work, see https://developers.google.com/cloud/sdk/crypto for details or consider using .json private key instead.
##[error]The process '/opt/hostedtoolcache/gcloud/270.0.0/x64/bin/gcloud' failed with exit code 1

Support .env files

TL;DR

Create .env file with secrets fetched

Design

steps:
- id: secrets
  uses: GoogleCloudPlatform/github-actions/get-secretmanager-secrets@master
  with:
    secrets: my-project/user-service
    dotenv: true
    overwriteFile: true

overwrite flag tells if we should remove existing .env file or not.

The following would happen:

• Fetch secrets for my-projects/user-service
• Remove .env file if overwrite === true
• Create .env with secrets obtained

gcloud.builds.submit

I am running the sample workflow and I am not able to get it working

Build step output

ERROR: (gcloud.builds.submit) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials, or if you have already logged in with a
different account:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.

I thought the auth is automatically inferrred from the gcloud setup step, do I need to do anything additional here?

gcloud.auth.activate-service-account Authentication Error

This workflow was working fine until day.

Nothing has been changed but I am now receiving this error:

ERROR: (gcloud.auth.activate-service-account) There was a problem refreshing your current auth tokens: invalid_grant: Invalid JWT Signature.

setup-gcloud: Using the project_id from the service_account_key.

TL;DR

Have a way (or by default) to use the project_id from the service_account_key.

Design

The service_account_key json should have a project_id value in it and it would be nice to be able to use that rather than having to pass in a separate project_id. Either by default or a flag to ask the action to use it. This would avoid having to have a separate secret for this.

Feature request: Add example flow for Cloud Build

Then the users can automatically build a container via Cloud Build even though they are not deploying it after each merge.

ERROR: (gcloud.builds.submit) Storage Object Get Access Error

I keep getting this issue

ERROR: (gcloud.builds.submit) HTTPError 403: <?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>*** does not have storage.objects.get access to <project-id>

I keep getting this error. These are the permission the service account has.

I just cannot figure this out. What am I missing?

Setup GCloud Using Python 3.x within container fails to find path.

Here is my workflow file for using it.
Notice this is running in a swift container from here

jobs:
  test:
    name: Integration Tests
    runs-on: ubuntu-latest

    container: 
      image: vapor/swift:5.2-bionic

    steps:
    - name: 'Checking out repo'
      uses: actions/checkout@v2
      
    - name: 'Installing Python'
      uses: actions/setup-python@v1
      with:
        python-version: '3.x'

    - name: 'Downloading and Setting up gcloud'
      uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
        service_account_email: ${{secrets.EMAIL}}
        service_account_key: ${{secrets.KEY}}

I'm getting the following error

Any pointers or commands I can try maybe setting the path or something else?

Documentation request: how to make a custom IAM role for Cloud Run with the most restrictive permissions

Quoting https://github.com/GoogleCloudPlatform/github-actions/blob/master/example-workflows/cloud-run/README.md

https://github.com/GoogleCloudPlatform/github-actions/blob/20c294aabd5331f9f7b8a26e6075d41c31ce5e0d/example-workflows/cloud-run/README.md#L61-L64

I have tried and failed to figure out how to do this. Official documentation on how to create an IAM role with the most restrictive permissions possible to deploy to Cloud Run (or ideally a specific service on Cloud Run) would be incredibly useful.

Bypass authentication in setup-gcloud

Hi,

I'm using the setup-gcloud Github action to setup a Cloud Pub/Sub Emulator in my workflow, for which I theoretically don't need to authenticate. I would like to avoid creating a dummy service account just for that purpose.

Is there a way to bypass the authentication part of setup-gcloud? I tried to pass a dummy key but I get the following error:

Run GoogleCloudPlatform/github-actions/setup-gcloud@master16s
##[error]Node run failed with exit code 1
Run GoogleCloudPlatform/github-actions/setup-gcloud@master
gcloud auth activate-service-account --key-file=/tmp/tmp-2698ACb7IjACbz1P
/bin/tar xz -C /home/runner/work/_temp/1af810e3-a86c-4475-a434-53d924a4922d -f /home/runner/work/_temp/8f778557-1f43-4cab-b7b5-e77550f7f01c
ERROR: (gcloud.auth.activate-service-account) Missing required argument [ACCOUNT]: An account is required when using .p12 keys

Maybe a setting making the authentication optional would work?

Thanks!

What is the release cycle for this project?

Question

It's been over a month since setting a project_id has been added to setup-gcloud. I was wondering when the next release would be cut as that's a feature I'd like to have but don't want to use master in production projects.

PyOpenSSL is not available

Hello!

I was trying to deploy an application to Google App Engine.
I get the workflow from the documentation example:

deploy:
  name: Deploy to Google App Engine

  runs-on: ubuntu-latest

  steps:
    - uses: actions/checkout@v1

    - name: Setup Google Cloud Platform
      uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
        version: '273.0.0'
        service_account_email: ${{ secrets.GCP_SA_EMAIL }}
        service_account_key: ${{ secrets.GCP_SA_KEY }}

    - run: gcloud info

But I get this error:

Run GoogleCloudPlatform/github-actions/setup-gcloud@master
gcloud auth activate-service-account *** --key-file=/tmp/tmp-17045ATDT0qazBcZQ
/bin/tar xz -C /home/runner/work/_temp/692985c1-383b-42c2-b29a-37c2cde1bad8 -f /home/runner/work/_temp/9e89d332-df66-4ffe-9c86-fb577f3c514d
WARNING: .p12 service account keys are not recomended unless it is necessary for backwards compatability. Please switch to a newer .json service account key for this account.
ERROR: (gcloud.auth.activate-service-account) PyOpenSSL is not available. If you have already installed PyOpenSSL, you will need to enable site packages by setting the environment variable CLOUDSDK_PYTHON_SITEPACKAGES to 1. If that does not work, see https://developers.google.com/cloud/sdk/crypto for details or consider using .json private key instead.
##[error]The process 'gcloud' failed with exit code 1
##[error]Node run failed with exit code 1

Add a "build and deploy to Cloud Run" example workflow

GitHub Actions are an ideal way to automatically build and deploy to Cloud Run.

I suggest we add an example workflow

There already are a lot of such third party tutorials:

• Deploying projects to Cloud Run using GitHub Actions
• Publish your Cloud Run App with GitHub Actions
• Automate Cloud Run deployment in a minute (with GitHub actions)
• Github Action for Google Cloud Run

Feature request: Google Secret Manager example/support

Hi, I just recently found out about this, I hope we can add an example in this repo. Especially when Github Actions doesn't support org-level secrets yet.

Some inspiration from azure key vault:

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # checkout the repo
    - uses: actions/checkout@master
    - uses: Azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }} # Define secret variable in repository settings as per action documentation
    - uses: Azure/[email protected]
      with:
        keyvault: "myKeyVault"
        secrets: 'mySecret1, mySecret2'
      id: myGetSecretAction
    - uses: Azure/docker-login@v1
      with:
        login-server: mycontainer.azurecr.io
        username: ${{ steps.myGetSecretAction.outputs.mySecret1 }}
        password: ${{ steps.myGetSecretAction.outputs.mySecret2 }}
    - run: |
        cd go-sample
        docker build . -t my.azurecr.io/myimage:${{ github.sha }}
        docker push my.azurecr.io/myimage:${{ github.sha }}
        cd ..

Where you can get the outputs via steps?

cc @sethvargo

setup-gcloud: authentication fields description is misleading

TL;DR

The documentation for setup-gcloud description is misleading:

If service_account_key is specified, authenticates the gcloud CLI tool
using the inputs: service_account_email and service_account_key. Please
see the [Service Account documentation][sa-iam-docs] for more information.

service_account_email is listed first, however in fact only service_account_key is used, as email is pulled from the key (#8, #9).

Expected behavior

The documentation must clearly reflect that only service_account_key is needed for authentication, and also outline the specific circumstances when service_account_email should be used.

docker-credential-gcloud + gcloud 298.0.0 + ubuntu-18.04: "OPENSSL_1_1_1 not found" error

TL;DR

Using GoogleCloudPlatform/github-actions/setup-gcloud@master with ubuntu-latest. Builds suddenly started failing overnight due to docker-credential-gcloud throwing a version OPENSSL_1_1_1' not found error.

Expected behavior

docker-credentials-gcloud should allow me to pull an image from my gcp repo

Observed behavior

docker-credentials-gcloud fails complaining about version OPENSSL_1_1_1' not found and build fails.

https://github.com/austinpray/kaori/pull/134/checks?check_run_id=814820107#step:6:90

ERROR: gcloud failed to load: /tmp/_MEIhQjalI/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so)

👉 Reverting to version 297.0.1 as in austinpray/kaori#133 makes the build pass again.

Reproduction

Here is me fixing it by pinning the gcloud version to an older one: austinpray/kaori#133

Here is me reproducing the issue by unpinning the version: austinpray/kaori#134 (https://github.com/austinpray/kaori/pull/134/checks?check_run_id=814820107#step:6:90)

Current runner version: '2.263.0'
Operating System
  Ubuntu
  18.04.4
  LTS
Virtual Environment
  Environment: ubuntu-18.04
  Version: 20200621.1
  Included Software: https://github.com/actions/virtual-environments/blob/ubuntu18/20200621.1/images/linux/Ubuntu1804-README.md

Action YAML

name: CI

on:
  push:
    branches: [ master ]
    paths-ignore:
      - workloads/**
      - fluxcd/**
  pull_request:

jobs:

  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
        project_id: ${{ secrets.GCLOUD_PROJECT_ID }}
        service_account_key: ${{secrets.GOOGLE_APPLICATION_CREDENTIALS}}
        export_default_credentials: true
    - run: gcloud auth configure-docker
    - name: Create a .env file
      env:
        SLACK_API_TOKEN: ${{secrets.SLACK_API_TOKEN}}
        SLACK_SIGNING_SECRET: ${{secrets.SLACK_SIGNING_SECRET}}
        SLACK_VERIFICATION_TOKEN: ${{secrets.SLACK_VERIFICATION_TOKEN}}
        GCLOUD_SERVICE_ACCOUNT_INFO: ${{secrets.GCLOUD_SERVICE_ACCOUNT_INFO}}
        IMAGES_BUCKET_GCLOUD: ${{secrets.IMAGES_BUCKET_GCLOUD}}
      run: ./scripts/create-dotenv-from.py .env.example > .env
    - name: Build the Docker image
      run: |
        docker pull $CACHE_FROM_IMAGE || true
        make
      env:
        CACHE_FROM_IMAGE: us.gcr.io/${{secrets.GCLOUD_PROJECT_ID}}/kaori/kaori:master
    - name: Run the tests
      run: ./tests/run.sh
    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v1
      with:
        token: ${{ secrets.CODECOV_TOKEN }}
    - name: Run the card sim
      run: make test-sims
    - name: Push to GCR
      if: github.ref == 'refs/heads/master'
      run: |
        docker tag austinpray/kaori/kaori us.gcr.io/${{secrets.GCLOUD_PROJECT_ID}}/kaori/kaori:master-${{github.sha}}
        docker tag austinpray/kaori/kaori us.gcr.io/${{secrets.GCLOUD_PROJECT_ID}}/kaori/kaori:master
        docker push us.gcr.io/${{secrets.GCLOUD_PROJECT_ID}}/kaori/kaori:master
        docker push us.gcr.io/${{secrets.GCLOUD_PROJECT_ID}}/kaori/kaori:master-${{github.sha}}

Repository

austinpray/kaori#133

If you want temporary access to this repo for debugging and such: I can give it to you. This is just a silly side project.

Application default credentials not set

I tried using this step right before using Pulumi to orchestrate some infrastructure on GCP. However, Pulumi's whining about ADC:

 +  pulumi:pulumi:Stack infrastructure-develop create 
    gcp:storage:Bucket my-bucket-eO4o4P1qpwmyq4U7Mk5Jq27FTj3t08IZ  error: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
 +  pulumi:pulumi:Stack infrastructure-develop create 
    gcp:storage:Bucket my-bucket-eO4o4P1qpwmyq4U7Mk5Jq27FTj3t08IZ  1 error
 
Diagnostics:
  gcp:storage:Bucket (my-bucket-eO4o4P1qpwmyq4U7Mk5Jq27FTj3t08IZ):
    error: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

I assumed that using this action as a setup step before using Pulumi would work, but apparently that's not the case. Any ideas?

Include alpha and beta components out of the box

For FINERACT-970, I am (need to) use gcloud beta builds triggers run XYZ in my YAML using this GitHub Action, but this currently fails due to "beta components" not being installed - and their automatic installation apparently not "just" working out of the box.

I'll see if I can fix this just by adding --quiet, but would there be any downside to just always including "beta components" in the image of this action? -- Or could/should --quiet perhaps by made true by default in this image?

#103 seems to be somewhat related to this.

setup-gcloud does not work in example

it appears as though the setup-gcloud action in the example does not work. The workflow example cited in this repository and on the marketplace uses ubuntu-latest as the image, but its missing python-openssl from the base image as well as a setting . To make it work, this repository either needs to include it itself (probably not considering how sparse this is) or you need to do the following

...
env:
    CLOUDSDK_PYTHON_SITEPACKAGES: 1

...

    # steps
 
    - name: install python-openssl
       run: sudo apt-get install -y python-openssl -o=Dpkg::Use-Pty=0

Error from server (NotFound): deployments.extensions "xxxxxx" not found (pipeline got response fail, but the deployment is successful)

TL;DR

in 'github-actions/example-workflows/gke/.github/workflows/gke.yml'
line 77-78:

        ./kustomize build . | kubectl apply -f -
        kubectl rollout status deployment/$IMAGE

you use the pipeline, you can't use 'deployment/$IMAGE' to get back.
so it will be a pipeline error, and you got
"Error from server (NotFound): deployments.extensions "xxxxxx" not found"
Expected behavior

no errors with 'deploy', if I change it to 'get deployments', I got

deployment.apps/gke-test configured
NAME       READY   UP-TO-DATE   AVAILABLE   AGE
gke-test   0/1     1            0           20m

and finished, so it is the only problem.
Observed behavior

Reproduction

Action YAML

# Paste your complete GitHub Actions YAML here, removing
# any sensitive values.

Repository

Additional information

Need of using base64 missing in GKE example documentation

TL;DR

In the GKE example readme file should be clarified that the GKE_SA_KEY has to contain the base64 enconding of the JSON key file and not the plain file

Storage Object Get Access Error

TL;DR

When using the cloudbuild workflow, the action job reports as failing due to a storage access error. However the job is correctly triggered in cloudbuild and completes successfully.

My problem seems similar to the one described in #49, but as I needed some clarification I opened this issue.

Expected behavior

The workflow would complete successfully

Observed behavior

The error message:

Build and push image to Google Container Registry(4s)

Run gcloud builds submit \
  gcloud builds submit \
    --quiet \
    --tag "gcr.io/$PROJECT_ID/$REPOSITORY_NAME:$GITHUB_SHA"
  shell: /bin/bash -e {0}
  env:
    PROJECT_ID: ***
    CLOUDSDK_CORE_PROJECT: ***
    REPOSITORY_NAME: ***
    CLOUDSDK_METRICS_ENVIRONMENT: github-actions-setup-gcloud
Creating temporary tarball archive of 148 file(s) totalling 8.9 MiB before compression.
Some files were not included in the source upload.

Check the gcloud log [/home/runner/.config/gcloud/logs/2020.05.15/21.07.45.085431.log] to see which files and the contents of the
default gcloudignore file used (see `$ gcloud topic gcloudignore` to learn
more).

Uploading tarball of [.] to [gs://***_cloudbuild/source/1589576865.25-e65b89df2a91419fbff076630958d5ee.tgz]
Created [https://cloudbuild.googleapis.com/v1/projects/***/builds/59a1f2ff-beee-4f1a-8147-504efe4014fd].
Logs are available at [https://console.cloud.google.com/cloud-build/builds/59a1f2ff-beee-4f1a-8147-504efe4014fd?project=192068846044].
ERROR: (gcloud.builds.submit) HTTPError 403: <?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>*****@*****.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.</Details></Error>
##[error]Process completed with exit code 1.

Following the logs link I can see that everything ran fine in spite of the error:

Reproduction

Action YAML

name: ci

on:
  pull_request:
    types:
      - opened
      - synchronize
      - reopened
  push:
    branches:
      - master
    tags:
      - "[0-9]+.[0-9]+.[0-9]+"
env:
  PROJECT_ID: ${{ secrets.PROJECT_ID }}
  CLOUDSDK_CORE_PROJECT: ${{ secrets.PROJECT_ID }}

jobs:
  check:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Retrieve the repository name
        run: echo ::set-env name=REPOSITORY_NAME::$(echo "$GITHUB_REPOSITORY" | awk -F / '{print $2}')
        shell: bash
      - name: setup gcloud CLI
        uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
        with:
          service_account_key: ${{ secrets.GCP_SA_KEY }}
          project_id: ${{ secrets.PROJECT_ID }}
      - name: Build and push image to Google Container Registry
        run: |-
          gcloud builds submit \
            --quiet \
            --tag "gcr.io/$PROJECT_ID/$REPOSITORY_NAME:$GITHUB_SHA"

• I have a dedicated service account for my project
• I use a JSON key to authenticate
• I assigned it the following roles to this service account:
  • roles/cloudbuild.builds.builder
  • roles/cloudbuild.serviceAgent
  • roles/compute.serviceAgent
  • roles/container.clusterAdmin
  • roles/container.serviceAgent
  • roles/storage.admin

Additional information

As a workaround, I added a JSON key to the service account which got automatically created by GCP ([email protected]), use it to authenticate this action and it worked like a charm.

EDIT(June 21st 2020):

• My service account did not have the role roles/viewer.

Why not use "kubectl apply -k" ?

In the following line you install kustomize to build your kustomize templates but since kubectl 1.14 they started kustomize integration. is there any reasons why you don't use it?

https://github.com/GoogleCloudPlatform/github-actions/blob/master/example-workflows/gke/.github/workflows/gke.yml#L75-L85

Feature request: setup-gcloud should support installing optional components

TL;DR

The setup-gcloud action should support installing optional components. Currently, the only way to install optional components is to add a run action to your ci.yml like:

steps:
  - uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
  - name: Install Google Cloud SDK components
    run: yes | gcloud components install beta cloud-firestore-emulator

In practice, this approach seems to take a long time to execute, and often flakes.

[Proposal] Deploy to App Engine

I will makes use of the already needed setup-gcloud (wrap the gcloud command) in order not to recreate the features of gcloud:

• Uploading files to GCS
• Authentication
• Yaml parsing
• Creating and managing the GAE application, services and versions (deployment)
• Flags
• Java specifics (deploying appengine-web.xml, jars)

Prioritized functionality (gcloud flags, see inputs below)

• Deployables (app.yaml, cron, index …)
• Promote
• Version name
• Image url (for flex)

Deployable types: (Argument for not recreating gcloud)

• Single: app.yaml
• Java8: appengine-web.xml
• Java11: my_jar.jar
• Multiple apps: ~/my_app/app.yaml ~/my_app/another_service.yaml
• Multiple configs: app.yaml, index.yaml

Authentication methods:

• Exported credentials from setup-gcloud action
• Github secret with service account JSON
• Hosted runner

IAM Roles implications (based on comparison)

• App Engine Deployer (minimal config) - deploys but can not promote
• App Engine Admin (maximum config) - All the power
• App Engine Service Admin - Can not deploy but can migrate/start/stop traffic
  Note: Storage Admin role and Cloud Build Editor role are needed to use gcloud commands
  Note: An owner will be needed to create the App Engine application

example action.yaml

name: 'Deploy to Google App Engine'
description: 'Deploy to Google App Engine'
author: 'GoogleCloudPlatform'

inputs:
  project_id:
    description: 'The GCP project ID'
    require: false
  deliverables:
    description: |-
      The yaml files for the services or configurations you want to deploy.
      If not given, defaults to app.yaml in the current directory. If that is
      not found, attempts to automatically generate necessary configuration
      files (such as app.yaml) in the current directory.'
    required: false
  image-url:
    description: |-
      Deploy with a specific Docker image. Docker url must be from one of the
      valid gcr hostnames.
    required: false
  version:
    description: |-
      The version of the app that will be created or replaced by this
      deployment. If you do not specify a version, one will be generated for
      you.'
    required: false
  promote:
    description: 'Promote the deployed version to receive all traffic.'
    default: true
    required: false

outputs:
  url:
    description: 'URL of your App Engine Application'

runs:
  using: node12
  main: dist/index.js

Other notes:

• There is no GAE Admin Node.js library. Having a library or using a REST call would need additional work to copy the files over the GCS + additional steps.

gsutil is not working on windows

We are using setup-gcloud. It works perfectly on linux.
However on windows-latest we are having issues with gsutil not being found while gcloud is working.

This is the config that always returns an error

jobs:
  release-win-to-gcs:
    name: Release Win to GCP

    runs-on: windows-latest

    steps:
      - name: Setup gcloud CLI for Dev
        uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
        with:
          version: 'latest'
          project_id: ${{ env.PROJECT_ID }}
          service_account_email: ${{ env.SERVICE_ACCOUNT_EMAIL }}
          service_account_key: ${{ secrets.DEV_GITHUB_CI_CD_SA_KEY}}
      - run: gcloud info

      - run: gsutil

Error:

Looks like gsutil it's looking python in the wrong place.

.p12 service account keys PyOpenSSL errors

Hi, I've been trying to set this up and have been unsuccessful for several hours now. I've followed the documentation exactly like this:

# Setup gcloud CLI
      - uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
        with:
          version: '275.0.0'
          service_account_email: ${{ secrets.SA_EMAIL }}
          service_account_key: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS}}

Where the two secrets are provided in the environment using GitHub Secrets and the GOOGLE_APPLICATION_CREDENTIALS is the base64 encoding of the JSON secret key created for the service account. The service account is activated and I believe it has the right permissions, but the actions workflow fails every time with:

/bin/tar xz -C /home/runner/work/_temp/d808259f-fc3f-4628-8c8f-d8d611055a81 -f /home/runner/work/_temp/097fc0ed-8610-4d65-8481-6ae4de38eb3b
gcloud auth activate-service-account $GOOGLE_SERVICE_ACCOUNT_EMAIL --key-file=/tmp/tmp-7195qsuIHwy3Vvhx
WARNING: .p12 service account keys are not recomended unless it is necessary for backwards compatability. Please switch to a newer .json service account key for this account.
ERROR: (gcloud.auth.activate-service-account) PyOpenSSL is not available. If you have already installed PyOpenSSL, you will need to enable site packages by setting the environment variable CLOUDSDK_PYTHON_SITEPACKAGES to 1. If that does not work, see https://developers.google.com/cloud/sdk/crypto for details or consider using .json private key instead.
##[error]The process 'gcloud' failed with exit code 1

I've even tried installing PyOpenSSL but got the same error. I've also looked at the other 2 or 3 closed issues here with similar errors but didn't find any solution yet.

I'm not sure if I missed some documentation steps, but I am really at a dead-end to figure out what I am missing. Thanks for any advice you may have!

Latest commit causing action to fail

Hi there,

I'm using this to authenticate and deploy my app onto Google Cloud Run. Snippet below is from my github workflow file:

- uses: GoogleCloudPlatform/github-actions/setup-gcloud@master

It's started immediately failing, I believe due to the latest commit. I re-ran passed workflows from yesterday which now fail. The error on these and all new workflows is:

Run GoogleCloudPlatform/github-actions/setup-gcloud@master
/home/runner/work/_actions/GoogleCloudPlatform/github-actions/master/setup-gcloud/dist/index.js:7860
        methods.forEach(function(m) {
        ^^^^^^^

SyntaxError: Unexpected identifier
    at Module._compile (internal/modules/cjs/loader.js:718:23)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:785:10)
    at Module.load (internal/modules/cjs/loader.js:641:32)
    at Function.Module._load (internal/modules/cjs/loader.js:556:12)
    at Function.Module.runMain (internal/modules/cjs/loader.js:837:10)
    at internal/main/run_main_module.js:17:11
##[error]Node run failed with exit code 1

Any ideas?

"gke-test" hardcoded in gke.yml

On line 86 of the gke example workflow:
https://github.com/GoogleCloudPlatform/github-actions/blob/99d5219b2ca42b90042bd5bfdc39cc5ece899a4f/example-workflows/gke/.github/workflows/gke.yml#L86

If this isn't changed it causes the error Error from server (NotFound): deployments.extensions "gke-test" not found during the Deploy step. I think gke-test should be changed to $IMAGE here, so that it respects the environment variable.

[setup-gcloud] already downloaded gcloud binary not found on second and subsequent run

TL;DR

I am running self hosted runner and Google Cloud SDK is NOT installed on system. At first run setup-gcloud downloads GoogleCloud SDK and adds gcloud binary location to PATH. Everything works fine. On second run, gcloud cache is found, but gcloud location is not added to PATH. It results in error.

On Github's runner it runs fine, probably because Google Cloud SDK is installed on their VMs system-wide.

Expected behavior
Already cached gcloud is added to PATH and action runs fine.

Observed behavior

::debug::isExplicit: 286.0.0
::debug::explicit? true
::debug::checking cache: /home/actions/cache/gcloud/286.0.0/x64
::debug::Found tool in cache gcloud 286.0.0 x64
::error::Unable to locate executable file: gcloud. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

Reproduction

Action YAML

# Paste your complete GitHub Actions YAML here, removing
# any sensitive values.

Repository

Additional information
Running on self-hosted runner.

Missing permission for cloud run build

Using the four permissions mentioned in the README here my gcloud build is failing:

gcloud builds submit -t gcr.io/$PROJECT_ID/$SERVICE_NAME:$GITHUB_SHA

(...)

ERROR: (gcloud.builds.submit) HTTPError 403: <?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>*** does not have storage.objects.get access to 43928094282.cloudbuild-logs.googleusercontent.com/log-6a7876f9-9f49-478b-b055-12c290cc3c84.txt.</Details></Error>
##[error]Process completed with exit code 1.

can't build image with cloud-run

Question

I'm using this action to build and push my image, but got error below at step - name: Build

Run gcloud builds submit \
Creating temporary tarball archive of 13 file(s) totalling 11.3 KiB before compression.
Some files were not included in the source upload.

Check the gcloud log [/home/runner/.config/gcloud/logs/2020.07.06/05.12.34.092568.log] to see which files and the contents of the
default gcloudignore file used (see `$ gcloud topic gcloudignore` to learn
more).

Uploading tarball of [.] to [gs://***_cloudbuild/source/1594012354.217488-352bbe5e5f6542d7a3f55b23ea39924d.tgz]
ERROR: (gcloud.builds.submit) INVALID_ARGUMENT: invalid build: invalid image name "gcr.io/***/***:ef88a8...github_sha": could not parse reference: gcr.io/***/***:ef88a8...github_sha
##[error]Process completed with exit code 1.

I can't find any information about this error with Google.

How can I solve this?

Getting no credentials provided

Question

Trying to setup github action for my private repo and here is the step for setting up gcloud

- name: Setup Gcloud
      uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
        version: '275.0.0'
        service_account_key: ${{ secrets.GCLOUD_KEY }}
        export_default_credentials: true

I always get the following output

Seems like it never recognizes service_account_key is provided. Any insights or help is appreciated.

setup-gcloud - Automatically set project from service account if not specified

TL;DR

If the user sets a service_account, I think this demonstrates clear intent to use it with a project. If they don't set the project explicitly, we leave it unset. Since all service accounts must be associated with a project, would be convenient, and almost expected, to set the project as the project that the service account belongs to.

Design

If project_id is not set, and service account is set, set the project to the service account's project.

Yaml would have one less line required.

Add metrics tracking env var

We should modify the installer to set an user agent via an environment variable which will allow us to aggregate usage metrics for the setup-gcloud action.

GCP Installing Component Docker Version is outdated

This is probably the wrong place but if i want to install a component which currently doesnt exist in the supplied gcp sdk
i.e:
gcloud auth configure-docker gcloud components install docker-credential-gcr docker-credential-gcr configure-docker

I get an error that my docker version is outdated. (Only supported docker version x >= 10.5)

I fixed this by adding another step:
- name: Install Latest Docker run: | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" sudo apt-get update sudo apt-get install docker-ce

However this approach cost around 1,5 minutes of time. Which is not really optimal.

I think this issue involves this one:
https://github.community/t5/GitHub-Actions/What-really-is-docker-3-0-6/td-p/30752

Maybe you have a better idea how to fix this.

Cheers

Cloud Run example is duplicated

There are 3 copies of cloud-run.yml, and they all differ slightly. Which one's the real one?

https://github.com/GoogleCloudPlatform/github-actions/blob/9025ea6d30e6ede8a4ef3673e9c694e7359e4f56/example-workflows/cloud-run/cloud-run.yml
https://github.com/GoogleCloudPlatform/github-actions/blob/9025ea6d30e6ede8a4ef3673e9c694e7359e4f56/example-workflows/cloud-run/workflows/cloud-run.yml
https://github.com/GoogleCloudPlatform/github-actions/blob/9025ea6d30e6ede8a4ef3673e9c694e7359e4f56/example-workflows/cloud-run/.github/workflows/cloud-run.yml

setup-gcloud should accept both base64-encoded keys and non-encoded keys

TL;DR

The setup-gcloud action requires base64-encoded keys as its service_account_key parameter, while most other actions (e.g. get-secretmanager-secrets) will accept both base64-encoded and non-encoded JSON keys with the documentation specifically asking for non-encoded keys.

Design

I can think of a couple possible solutions to this issue. One would be for setup-gcloud to detect whether the key is base64-encoded or not as some of the other actions do. Another would be to provide two separate input parameters, one that accepts base64-encoded values and one that expects the value to not be encoded.

Resources

https://github.com/GoogleCloudPlatform/github-actions/blob/master/setup-gcloud/README.md

https://github.com/GoogleCloudPlatform/github-actions/blob/master/get-secretmanager-secrets/README.md

https://github.com/GoogleCloudPlatform/github-actions/blob/master/get-secretmanager-secrets/src/client.ts#L49-L55

Feature request: actions for working with cloud storage

In my projects, I often have to build and publish documentation as static websites. I typically host these files with Cloud Storage, so having an easy to use action that takes a folder in my repository and uploads it to a given bucket.

Perhaps a simplified workflow could be something like:

name: Master workflow

on:
  push:
    branches:
      - master

jobs:
  build-and-publish-docs:
    name: Build and publish documentation
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1
      - name: Build documentation
        run: |
          do whatever to build the documentation
      - uses: ____/publish-to-gcs@v666
        with:
          source-folder: ./generated/docs/of/doom
          bucket-name: my-amazeballs-bucket
          object-key-prefix: /the-docs/1.0.1/
          clear-existing-files-first: true

cannot find latest release

I have:

build.yml

- uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
   with:
     version: '272.0.0'
     service_account_key: ${{ secrets.GOOGLE_CLOUD_SERVICE_ACCOUNT_JSON_KEY }}

When I look at the logs, I see:

(node:7088) UnhandledPromiseRejectionWarning: Error: Failed to find release, os: linux arch: x64 version: 272.0.0

Interestingly some of my commands do seem to run okay, though maybe it's the previously cached version?

Cloud run, which was made GA in this version, does not run:

build.yml

- name: Deploy
   run: gcloud run deploy --image gcr.io/"$GOOGLE_CLOUD_PROJECT"/"$CONTAINER_IMAGE":"$GITHUB_SHA" --platform managed

Result:

ERROR: (gcloud) Invalid choice: 'run'.
This command is available in one or more alternate release tracks.  Try:
  gcloud alpha run
  gcloud beta run

gsutil Support

@craigdbarber Is there any plan of having a gsutil action included in this repo? There doesn't seem to be anything now. This PR implements it but it was never merged and it looks like that repo wont exist after 12/31/2019. Thanks!

gsutil requires a project id

When running gsutil in an action.yml file, an error is thrown requiring a project id during configuration.

    - run: |
        # View storage
        gsutil ls

You are attempting to perform an operation that requires a project id, with none configured. Please re-run gsutil config and make sure to follow the instructions for finding and entering your default project id.

However if gcloud is activated with a valid project_id, then, the gsutil command works correctly. For example, like the following:

gcloud auth activate-service-account --project=${someProject_id} --key-file=${tmpKeyFilePath}`

The problem may be here https://github.com/GoogleCloudPlatform/github-actions/blob/2a9cbc618184da4718e0d16a1b0a055a4f90f66e/setup-gcloud/src/setup-gcloud.ts#L84

Note there is no --project, which could be picked up from the key-file.

AppEngine deploy tests cannot run in parallel

TL;DR

Conflicts due to same name being deployed.

Expected behavior
Tests pass

Observed behavior
Tests fail (example)

File upload done.
ERROR: (gcloud.app.deploy) ABORTED: Cannot operate on apps/***/services/default/versions/gcloud because an operation is already in progress for apps/***/services/default/versions/gcloud by 04ecd77a-b346-4d7f-bdbc-25c0b8139baf.
##[error]The process '/opt/hostedtoolcache/gcloud/295.0.0/x64/bin/gcloud' failed with exit code 1

Ability to specify latest gcloud version

While setting up the action you specify the version of gcloud to use, E.g. 174.0.0.1. Have the ability to specify latest version.

Example

steps:
- uses: actions/checkout@v1
- uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
  with:
      version: 'latest'
      service_account_email: ${{ secrets.GCP_SA_EMAIL }}
      service_account_key: ${{ secrets.GCP_SA_KEY }}
- run: gcloud info

If changing major versions may not be desirable without prior testing, perhaps a semver-type formatting to run at any patched or minor versions of the major version specified.

example: version: '^274.0.0'

export_default_credentials: true isn't setting $GOOGLE_APPLICATION_CREDENTIALS for later steps

TL;DR

I set export_default_credentials: true, and in later steps test failed due to missing credentials.

Expected behavior
$GOOGLE_APPLICATION_CREDENTIALS should be set with the credentials for all later steps.

Reproduction

Action YAML

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1
    - uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
       version: '270.0.0'
       service_account_email: ${{ secrets.GCP_SA_STG_EMAIL }}
       service_account_key: ${{ secrets.GCP_SA_STG_KEY }}
       export_default_credentials: true
    - name: set gcp cradentilas
      run:  gcloud info
    - name: echo gcp cradentilas
      run:  echo this is  ${GOOGLE_APPLICATION_CREDENTIALS}  
    - uses: actions/checkout@v2
    - name: Set up JDK 1.8
      uses: actions/setup-java@v1
      with:
        java-version: 1.8
    - name: Build with Maven
      run: mvn clean install --file pom.xml

Additional information
I submitted in the past some PR that fixed it, when I run same action with that PR branch it works.

5d5bbc introduces syntax error

There a syntax error on line 9124 of setup-gcloud/dist/index.js

let b64tab = function(bin) {
            let t = {};
            for (let i = 0, l = bin.length; i < l; i++) t[bin.charAt(i)] = i;
            return t;
          })(b64chars);

should be

let b64tab = function(bin) {
            let t = {};
            for (let i = 0, l = bin.length; i < l; i++) t[bin.charAt(i)] = i;
            return t;
          }(b64chars);

Error with macos runner

Hi, I have a problem to use setup-gcloud@master with macos runner in my workflow.
When I used it with ubuntu-latest runner, I succeed it. GCP_EMAIL and GCP_KEY of secrets are set at both situations.
Did I miss anything?

What I did:

runs-on: macos-latest
steps:
- uses: actions/checkout@v1

- name: setup gcloud
  uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
  with:
    version: '270.0.0'
    service_account_email: ${{ secrets.GCP_EMAIL }}
    service_account_key: ${{ secrets.GCP_KEY }}

What I received:

Run GoogleCloudPlatform/github-actions/setup-gcloud@master
gcloud auth activate-service-account *** --key-file=/var/folders/24/[...ellipsis]
##[error]There was an error when attempting to execute the process 'gcloud'. This may indicate the process failed to start. Error: spawn gcloud ENOENT
/usr/bin/tar xz -C /Users/runner/runners/2.163.1/work/_temp/[...ellipsis] -f /Users/runner/runners/2.163.1/work/_temp/[...ellipsis]
##[error]Node run failed with exit code 1

Thanks.

GAE example fails to deploy, missing non-interactive flag '--quiet' or '-q'

TL;DR

The GAE example fails to deploy because the cli app expects an interactive prompt.

https://github.com/GoogleCloudPlatform/github-actions/blob/master/example-workflows/gae/.github/workflows/app-engine.yml

Do you want to continue (Y/n)?  
ERROR: (gcloud.app.deploy) This prompt could not be answered because you are not in an interactive session.  You can re-run the command with the --quiet flag to accept default answers for all prompts.
##[error]Process completed with exit code 1.

Reproduction

https://github.com/powersjcb/monitor/runs/694617478

GitHub-hosted runners include Google Cloud SDK

When I read Github Documentation, I find Google Cloud SDK available in GitHub-hosted virtual environments. So, I tried write in my workflow, It's work very well.
Then the question comes, Why do we need this action?