cloud-native-toolkit / image-cli-tools

Container image that provides common tools used in Cloud Native Toolkit infrastructure

Cloud-Native Toolkit cli tools

This repository builds a Docker image whose container is a client for interacting with different cloud providers (IBM Cloud, AWS, Azure).

The container includes the following tools:

• terraform cli
• IBM Cloud cli
• AWS cli
• Azure cli
• bash
• kubectl cli
• oc cli
• git cli
• perl cli
• jq cli
• yq3 cli
• yq4 cli
• helm cli
• terragrunt cli

Warning: The material contained in this repository has not been thoroughly tested. Proceed with caution.

Getting started

Prerequisites

To run this image, the following tools are required:

• docker cli
• docker backend - Docker Desktop, colima, etc

Running the client

Start the client to use it.

• To run the toolkit container:

  docker run -itd --name toolkit quay.io/cloudnativetoolkit/cli-tools

Once the client is running in the background, use it by opening a shell in it.

• To use the toolkit container, exec shell into it:

  docker exec -it toolkit /bin/bash

  Your terminal is now in the container.

Use this shell to run commands using the installed tools and scripts.

When you're finished running commands, to exit the client.

• To leave the toolkit container shell, as with any shell:

  exit

  The container will keep running after you exit its shell.

If the client stops:

• To run the toolkit container again:

  docker start toolkit

The toolkit container is just a Docker container, so all Docker CLI commands work.

Development

Prerequisites

To use/build this image, the following tools are required:

• Docker - kinda obvious, but since we are building, testing and running a Docker image, you need to have the tool available
• node/npm - (optional) used to consolidate the configuration and scripts for working with the image, it is highly recommended that npm is used; however, it is possible to run the scripts directly by looking at package.json and providing the appropriate values

Using the image

To use the image, a local image of the tool is required. You can get the image either by pulling from Docker Hub or building locally:

npm run pull

OR

npm run build

After that, start the image in an interactive terminal with:

npm start

File Layout

• package.json - scripts and config for the image build
• Dockerfile - the docker image definition
• config.yaml - the test config file for the container-structure-test tool
• scripts/ - directory for shell scripts used by package.json scripts to build, test, and push the image
• src/ - directory containing files that should be included in the built image

image-cli-tools's Issues

Update readme

The readme is VERY old...

Maureen would like helm 3 to be included in the image

ArgoCD Bootstrap Application fails to Synchronize

I ran a test to deploy a BOM including OpenShift and ArgoCD. The test worked fine using Multipass, but fails when using Docker and the cli-tools container.

The OpenShift cluster is installed, but ArgoCD fails to synchronize the Bootstrap Application. It reports the following error:

Ignoring non-table value ([]) Error: template: default/charts/services/templates/project.yaml:1:26: executing "default/charts/services/templates/project.yaml" at <include "argocd.destinations" .Values.global.destinations>: error calling include: template: default/charts/applications/templates/_helpers.tpl:12:7: executing "argocd.destinations" at <.targetNamespace>: can't evaluate field targetNamespace in type interface {} Use --debug flag to render out invalid YAML

@binnes and I did some investigation. We found that when using the cli-tools container, the main/argocd/0-bootstrap/cluster/default/values.yaml file in GitOps is generated differently vs when using Multipass.

Using cli-tools, it is generated like this:

global:
  repoUrl: https://github.com/deleeuwblue/watsondemogitDocker.git
  targetRevision: main
  targetNamespace: openshift-gitops
  destinations:
    targetNamespace: '*'
  pathSuffix: cluster/default
  prefix: ''

Using Multipass, it is generated like this:

global:
  repoUrl: https://github.com/deleeuwblue/watsondemogit.git
  targetRevision: main
  targetNamespace: openshift-gitops
  destinations:
    - targetNamespace: '*'
  pathSuffix: cluster/default
  prefix: ''

Note there is a '-' missing ahead of 'targetNamespace' with cli-tools. When this is corrected in the GitOps file, ArgoCD is able to successfully synchronize the Bootstrap Application.

One possible cause could be a mismatch with the yq versions. With Multipass, the cli-tools.yaml installs v4.25.2. The cli-tools container includes v4.27.2.

Update the workflow to push the image to quay.io/cloudnativetoolkit/cli-tools

The quay.io/cloudnativetoolkit/cli-tools repo already exists and has been initially populated with the contents of ibmgaragecloud/cli-tools (using skopeo copy).

The workflow needs to be updated to use quay.io/cloudnativetoolkit/cli-tools as the base tag and to use the QUAY_CNTK_XXX secrets for the credentials.

Add image validation to the PR process

• When a PR is created the image should be built and unit tested to ensure there are no regressions

Move image build process to quay.io

Provide a way to save data to a named volume so it is not lost between image instances

Previously under Docker Desktop we were able to write files to a mounted host filesystem. Under colima we currently have filesystem permission issues and cannot reliably write to the directory. If we can't save the files to the host filesystem, we at least need a way to preserve the files (particularly the tfstate) between running container instances.

container image lacks ssh client

This container image lacks ssh which causes test actions that might require using ssh to no longer work as opposed to the container used in previous repo git-actions: quay.io/ibmgaragecloud/cli-tools:v0.15

docker run -it quay.io/cloudnativetoolkit/cli-tools:v1.1
~ $ ssh
/bin/sh: ssh: not found
~ $ /usr/bin/ssh
/bin/sh: /usr/bin/ssh: not found

Split image into four separate images

• Base cli-tools - jq, yq, oc, kubectl, etc
• ibmcloud - base + ibmcloud cli
• aws - base + aws cli
• azure - base + azure cli

jq & yq no longer available?

Following the instructions in the readme & it looks like jq & yq are no longer available in the cli-tools image...

I noticed this when executing a job inside a cluster using the v0.15 image, and jq is not available, despite the fact that I see it in the Dockerfile.

Upgrade v1.1 image to terraform 1.1.9

Tammy would like an arm version of the image

Update clis on v0.15 branch to latest

Error: The specified plugin cache dir /home/devops/.terraform.d/plugin-cache cannot be opened: stat /home/devops/.terraform.d/plugin-cache: no such file or directory

Related issue: cloud-native-toolkit/iascable#163

There are some problems with the CLI configuration:
╷
│ Error: The specified plugin cache dir /home/devops/.terraform.d/plugin-cache cannot be opened: stat /home/devops/.terraform.d/plugin-cache: no such file or directory
│
╵

The Lite branch should install 4.3 OC CLI and the IGC CLI

The Lite branch should install 4.3 OC CLI and the IGC CLI

Make the v1.1 branch the default branch

Will require an update to the workflow to tag v1.1 as "latest" as well