taitounited / taito-cli

An extensible toolkit for DevOps and NoOps

Taito CLI

Taito is a Finnish word meaning "ability" or "skill".

Taito command line interface is an extensible toolkit for developers and DevOps personnel. It defines a standard set of commands that can be used in any project no matter the technology or infrastructure. Thus, developers and DevOps personnel may always run the same familiar set of commands from project to project without thinking about the underlying infrastructure. This is made possible by shipping all required tools as a Docker container image, implementing the commands with plugins, and defining project specific settings in a configuration file. Continuous integration scripts also become more reusable and maintainable as they are based on the same set of commands and settings.

To get started see the website. For a command reference, see help.txt or run taito -h. For troubleshooting, see trouble.txt or run taito trouble.

Container images

Taito CLI container repository provides the following stable image optimized for local command line usage:

• cli: Taito CLI image to be used locally on command line. This is the default Docker image for Taito CLI. It includes tools for all major cloud providers.

Stable images optimized for CI/CD:

• ci: CI/CD image for building and testing.
• ci-dockerd: CI/CD image for building and testing. Includes also dockerd that is required in some CI/CD environments.
• ci-deploy: CI/CD image for deployment without any cloud provider specific tools.
• ci-aws: CI/CD image for deployment on Amazon Web Services.
• ci-azure: CI/CD image for deployment on Microsoft Azure.
• ci-gcp: CI/CD image for deployment on Google Cloud Platform.
• ci-do: CI/CD image for deployment on Digital Ocean.
• ci-openshift: CI/CD image for deployment on OpenShift.
• ci-all: CI/CD image that includes tools for all cloud providers.

TIP: If your CI/CD pulls the whole Docker image on each build, optimize the image pull by setting up caching on your CI/CD, or by using your own Docker registry as a mirror. You can also create a custom Docker image that includes only the tools you actually need.

TIP: Read this if you cannot use Taito CLI on your CI/CD or you want to avoid pulling a Taito CLI docker container image on each build.

Special purpose Docker images:

• all: Taito CLI image that contains all tools required either locally or on CI/CD. It is used for running the Taito CLI integration test suite.

Alternative versions for each image:

• *-canary: Canary version includes all the latest implemented features.
• *-dev: Development version includes the latest changes made during Taito CLI development.
• *-N.N.N: Docker image tagged with Taito CLI version number in case you need an older version of Taito CLI. Note that it is recommended to always use the latest stable Taito CLI release available instead of a tagged version.

Contributing

TODO: Something about Taito CLI vs external extensions.

Development

Website development

For website development, see the www/README.md.

Taito CLI development

Install Taito CLI normally using the installation instructions. You can run a taito command in development mode by using the -d, --dev flag (e.g. taito -d status). In the development mode your local taito-cli directory is mounted in the Taito CLI container. If you are working with your own fork, update your PATH or taito symlink so that it points to your forked version of the Taito CLI.

How to implement a command:

1. Fork taito-cli repository.
2. Add a new bash(.sh), python(.py) or javascript(.js) file to one of the plugin folders and make it executable with chmod +x FILE. Try to implement one of the Taito CLI prefined commands if it suits your purpose (see the help.txt).
3. Add unit tests for your command. You can execute a single unit test by executing the corresponding bats file. All unit tests are run automatically on git push and during CI build, but you can also run them manually with the taito unit command.
4. Make sure that the plugin description written in plugin README.md is up-to-date.
5. If you implemented some new plugin specific commands, add the command descriptions to the help.txt file of your plugin. Also, add the plugin specific commands to autocomplete.sh and descriptions.sh files located in root of plugins directory.
6. Add the plugin to plugins.md file if it is not there already.
7. Make a pull request.

For detailed instructions, see plugin development instructions on the docs and on the tutorial.

License

Taito CLI is licensed under the MIT license and maintained by Taito United.

taito-cli's Issues

Use a separate taito and admin user accounts for --admin implementation

minimal-template

Related to #27 (make)

Move commands to separate extensions

Leave only basic, link and template plugins in taito-cli. Move all other commands to separate extensions: taito-cli-project, taito-cli-misc.

NOTE: When ci mode is enabled, the extension probably should be downloaded to the project workspace instead of "home directory".

WordPress project template improvements

See https://github.com/TaitoUnited/wordpress-template/projects/1

Taito-cli extensions do not work

taito_home change made to taito file broke it (Local url).

Azure zone example hardening

Taito-cli editor plugins

Closing this issue as a duplicate. Atom plugin already exists and there is a separate issue for VS Code: #80

Google Cloud zone example

Bare zone commands

Google Cloud zone commands

AWS zone example

Advanced CI/CD workflows

• Manual review before deployment if certain files have been changed since the last deployment: CI/CD scripts, Kubernetes / Helm / Terraform templates.
• Send a direct message to the developer who broke the build.

For example: Jenkins for workflows + Google Container Builder for building. Or just Jenkins by itself on non-gcloud environments.

Install Jenkins on Kubernetes using Helm and/or Terraform: https://github.com/mcuadros/terraform-provider-helm, https://hub.kubeapps.com/charts/stable/jenkins

Bare zone example

Setup Kubernetes, Postgres and object storage clusters, and Jenkins, on virtual machines or bare-metal servers using e.g. the following technologies:

• Ansible
• Terraform
• Kubeadm
• Helm
• PAF

NOTE: We already have an working Postgres cluster using PAF.

React-native project template

Confirm execution of external commands that havent' been run before

Full-stack-template based project for Taito CLI integration tests

Create full-stack-template based project that is deployed each time changes are pushed to dev branch of Taito CLI. Include also additional Taito CLI integration tests in the CI/CD build script.

Google Cloud zone example hardening

Google Cloud blog articles with the Security/Identity label. Especially for Kubernetes:

• https://cloudplatform.googleblog.com/2018/03/kubernetes-engine-private-clusters-now.html
• https://cloudplatform.googleblog.com/2018/03/Kubernetes-Engine-network-policy-is-now-generally-available.html
• https://cloudplatform.googleblog.com/2018/03/exploring-container-security-an-overview.html
• https://cloudplatform.googleblog.com/2017/11/precious-cargo-securing-containers-with-Kubernetes-Engine-18.html

Support external extensions also in CI build

Function support as plugins

Provide function support as a one or two taito-cli plugins. Avoid vendor lock-in. Here are just some of the alternatives:

• Serverless
• OpenFaas
• Fission
• Kubeless
• fnproject.io
• OpenWhisk

Perhaps implement a Serverless plugin to get a wide function support for different platforms, and some other plugin to get a more tailored function support on Kubernetes.

Download taito extensions during CI/CD

Print taito-cli version number on build logs

print-commands (autocomplete) implementation for each plugin or extension

Just filter commands in plugins/autocomplete and plugins/descriptions scripts by the plugins included in taito_plugins config of the current project? Such implementation would probably work ok also in Taito CLI extensions.

docker support for minimal-template (was: tool-template)

Combine this with minimal-template? That is, minimal-template should include docker example for running e.g. python implementation in docker.

Old implementation already exists in https://github.com/TaitoUnited/tool-template

gcloud: CDN

• On taito zone apply: Create a storage bucket and configure CDN: https://stackoverflow.com/questions/39802631/is-google-cloud-storage-an-automagical-global-cdn
• On CI build: Publish all static assets to ci_static_assets_location.
• On nginx container boot (server-template): Add ci_static_assets_location prefix to all static asset urls of index.html before starting nginx.

Support db proxy access when running integration/e2e tests on CI build

Taito-cli integration test tuning

• Execute all tests against locally installed minikube and postgres.
• Execute tests using docker hub or travis instead of google container builder?

Related to #3

Backup data to another provider

Especially required for storage buckets as data is really gone if the whole bucket is deleted. Versioning doesn't help in that case, I think.

Add README.md for each plugin

Bare zone example hardening

npm library template

Also separate npm library template for react-native? (see react-sheltr)

AWS zone example hardening

kubectl: Support also customername-dev namespace

Already supported? At least test it.

AWS zone commands

Yarn, make and pipenv plugins

Add also a setting: run on host/container

windows support

Windows support has been implemented for both Docker for Windows and Docker Toolbox. However, it does not work fully yet. See installation page for more details: https://taitounited.github.io/taito-cli/docs/02-installation

Run commands as current user to avoid files owned by root

Related to issue #34

Multiple nginx-ingress setups

kubernetes/ingress-nginx#1976

Support for 'any' environment

For example test-stable, feat-myfeature, hotfix-myfix.

Also take future alternative environments in account (see issue #67).

Generic API gateway solution

Related to issues #7 and TaitoUnited/full-stack-template#1.

Istio might be a bit overkill for simple stuff, but keep an eye on them also, as they are working on integrating API gateways: https://github.com/istio/community/blob/master/WORKING-GROUPS.md#api-management

Ambassador:
https://www.getambassador.io/
https://www.getambassador.io/user-guide/with-istio

Taito-cli output improvements

• Improved explanations for each command
• Quiet mode: Do not print anything that is not necessary (piping)

helm: helm.yaml values substitute

postgres/mysql: Ignore env rotate if not db secrets present

Prepare for contributions

Add CONTRIBUTING.md and other GitHub requirements (see GitHub report)

Azure zone example

Terraform plugin

Supports also helm: https://github.com/mcuadros/terraform-provider-helm

ChatOps

Backup manual secrets

Azure zone commands

Code Climate or SonarQube?

Full-stack-template already includes Code Climate support, but it is run only locally, because Code Climate docker image is HUGE!

Minimize container size

Use alpine instead of jessie?