cloudnativedevops / demo

Simple demonstration app for 'Cloud Native Devops'

Cloud Native DevOps with Kubernetes

Welcome! This is the example code repository to accompany the book 'Cloud Native DevOps with Kubernetes', by John Arundel and Justin Domingus. Buy the book here:

• Amazon US

• Amazon UK

• O'Reilly

About the book

From the preface:

You'll learn what Kubernetes is, where it comes from, and what it means for the future of software development and operations. You'll learn how containers work, how to build and manage them, and how to design cloud native services and infrastructure.

You'll understand the trade-offs between building and hosting Kubernetes clusters yourself, and using managed services. You'll learn the capabilities, limitations, and pros and cons of popular Kubernetes installation tools such as kops, kubeadm, and Kubespray. You'll get an informed overview of the major managed Kubernetes offerings from the likes of Amazon, Google, and Microsoft.

You'll get hands-on practical experience of writing and deploying Kubernetes applications, configuring and operating Kubernetes clusters, and automating cloud infrastructure and deployments with tools such as Helm. You'll learn about Kubernetes support for security, authentication, and permissions, including Role-Based Access Control (RBAC), and best practices for securing containers and Kubernetes in production.

You'll learn how to set up continuous integration and deployment with Kubernetes, how to back up and restore data, how to test your cluster for conformance and reliability, how to monitor, trace, log, and aggregate metrics, and how to make your Kubernetes infrastructure scalable, resilient, and cost-effective.

The book aims to teach you everything you need to know to deploy, run, and scale applications in Kubernetes, and most importantly, to give you working example code for everything we demonstrate. That code is open source, available for free for you to use and adapt whether or not you buy the book. And here it is!

Show me the code

Almost all the example code involves our 'hello world' demo application. Here is the list of examples; follow the links to see the documentation on each example.

• Build and run the demo application locally (start here!)
• Deploy the app to Kubernetes using kubectl
• Deploy the app with Helm (or with Helm 3)
• Manage the app with Helmfile
• Set up a namespace, resource requests, and limits for the app
• Add config data to the app's environment
• Pass config data to the app's command line
• Mount a config file in the container at runtime
• Add secret data to the app's environment
• Mount a secrets file in the container
• Store encrypted secrets in the app repo, decrypted automatically on deploy
• Develop the app locally with Skaffold
• Build and deploy the app automatically with Drone
• Build and deploy the app automatically with Google Cloud Build

Terraform examples

We also include some Terraform code examples, to help you manage cloud resources with code. Unfortunately we didn't have space to discuss these in the book, but we hope they'll be useful to you anyway.

Google Cloud

• Create a cloud storage bucket in code
• Create a cloud database instance in code
• Create a Kubernetes cluster in code

Amazon AWS

• Create a cloud storage bucket in code
• Create a cloud database instance in code
• Create a cloud virtual machine instance in code

You will need

To build and run all of these examples, you will need:

• Go (any recent version is fine)
• Docker version 18.03 or above

Where you need other tools for specific examples, we'll mention that in the README for the example.

Contributing to the repo

We would absolutely love it if you contributed! Feel free to send us a PR to add new examples, add versions of the examples for different cloud providers (for example Microsoft Azure), or fix or improve the existing examples.

Known Issues

apiVersion

When we released the book most k8s clusters still used extensions/v1beta1 for Deployments. On newer versions of k8s Deployment has been moved to apiVersion: apps/v1. If you get the message:

error: unable to recognize "k8s/deployment.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"

when trying out the examples then try updating apiVersion: extensions/v1beta1 to apiVersion: apps/v1 in your deployment.yaml file(s).

Thanks to @thescott for pointing this out.

Helm 3

At the time of publishing Helm 3 was not yet released and we included the Helm 2 examples that included the additional steps of installing tiller.

Helm 3 is now released and tiller is no longer required.

We have added the hello-helm3 examples here with updated instructions if you would like to use the latest version of Helm.

You can read more about the changes between version 2 and 3 here.

kubectl run and Pod/Deployment v1.18 change

In the 1.18 release of Kubernetes the kubectl run command changed from creating a Deployment by default to creating a Pod instead.

We have a few examples where we use kubectl run to get familiar with running a container in k8s.

Later we discuss why using the declarative kubectl apply -f... is preferred over the imperative create, edit or run, because your version-controlled YAML files always reflect the real state of the cluster.

In our kubectl run example we show the output as deployment.apps "demo" created but on version 1.18 instead you will instead see pod/demo created.

The subsequent port-forward example would instead be: kubectl port-forward pod/demo 9999:8888

Again, using kubectl apply -f... and keeping your manifests tracked in source control is a better long-term solution.

Service port 8888 VS 9999

Depending on which version of the book you read you may see reference to using port 9999 for the service port here. This caused some confusion between the pod port, service port, and the port-forwarding port in the examples, so was changed in the latest revision to use 8888 for both the pod and the serivce. Thanks to @randoljt for catching this and sorry for any confusion.

demo's Issues

safaribooksonline.com code examples

Hi,
I'm reading the online version of "Cloud Native DevOps with Kubernetes" at safaribooksonline.com and seems like the code examples do not correspond with the same code in this repo. For example, in Chapter 12 the service definition name is {{ .Values.container.name }}-service-{{ .Values.environment }} but in this repo is name: {{ .Values.container.name }}-service and the latter will fail when feeding the staging values yml file.

Chapter 2 kubectl deployment doesn't work on a real cluster

In chapter 2 the command you show to deploy the myhello application gives a different output than you show when I run it on the master of a 3 node cluster:

$ kubectl run demo --image=dwschulze/myhello --port=9999 --labels app=demo
pod/demo created

The output you show is:

deployment.apps "demo" created

When I try to do the port forwarding it fails because I have no deployments.

Do I need a deployment.yaml to deploy this onto a real cluster, or is there a different kubectl command I use?

error: unable to recognize "k8s/deployment.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"

May i know how to resolve this error from

I tried this code..

sed -i s#IMAGE_HERE#gcr.io/$GOOGLE_CLOUD_PROJECT/valkyrie-app:v0.0.1#g k8s/deployment.yaml

gcloud container clusters get-credentials valkyrie-dev --zone us-east1-d

kubectl create -f k8s/deployment.yaml

kubectl create -f k8s/service.yaml

And ended up receiving this error below

error: unable to recognize "k8s/deployment.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"

Screenshot below

Somwhat confusing port mapping in example

Hi, this just a minor comment. For clarity in the example hello-k8s, I recommend to have separate port numbers for container, service, and local port, like:

containerPort: 8888 (deployment.yaml)
port: 9999 (service.yaml)
and then issue the command:
# kubectl port-forward service/demo 19999:9999
Forwarding from 127.0.0.1:19999 -> 8888
Forwarding from [::1]:19999 -> 8888
Handling connection for 19999
Handling connection for 19999

I also noticed another confusing thing regarding the resulting output. The service port number 9999 is not mentioned at all in the log output. However, that's another story and outside of this scope :-)

In addition, on page 62, the service port is set to 9999, but the port-forwarding command refers to 8888. The results in an error:

# kubectl port-forward service/demo 9999:8888
error: Service demo does not have a service port 8888

Known issue guide

Hi,
Firstly thanks for writing this great book.
In the known issue part of readme, I found the resolution is not correct. Please correct it.
when trying out the examples then try updating apiVersion: extensions/v1beta1 to apiVersion: extensions/v1beta1 in your deployment.yaml file(s).

hello-helm example fails to install

I am following along with the examples in the book. When I try to issue the helm install command from the hello-helm/ example, I receive the following error:

Error: release demo failed: Deployment.apps "demo" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{"app":"demo", "environment":"development"}: `selector` does not match template `labels`]

The release is then listed as failed:

$ helm list
NAME	REVISION	UPDATED                 	STATUS	CHART     	APP VERSION	NAMESPACE
demo	1       	Fri Nov  1 13:43:34 2019	FAILED	demo-1.0.1	           	default 

Complete output:

$ kubectl get all --selector app=demo
No resources found.
$ helm install --name demo ./k8s/demo
Error: release demo failed: Deployment.apps "demo" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{"app":"demo", "environment":"development"}: `selector` does not match template `labels`]

The versions I run:

$ helm version
Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:47:22Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:39:02Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
$ docker version
Client: Docker Engine - Community
 Version:           19.03.4
 API version:       1.40
 Go version:        go1.12.10
 Git commit:        9013bf5
 Built:             Thu Oct 17 23:44:48 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.4
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.10
  Git commit:       9013bf5
  Built:            Thu Oct 17 23:50:38 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

I'm using MacOS 10.14.6 and the Kubernetes cluster set up by docker-desktop.

kubectl port-forward gives "error upgrading connection: error dialing backend: dial tcp 10.204.101.149:10250: i/o timeout"

I apply the .yaml files like this:

$ kubectl apply -f src/k8s/cloudnativedevops/ch04/deployment.yaml
deployment.apps/demo created

$ kubectl apply -f src/k8s/cloudnativedevops/ch04/service.yaml
service/demo created

Then when I try to start port forwarding it gives the error below:
$ kubectl port-forward --address 0.0.0.0 service/demo 6817:8888
error: error upgrading connection: error dialing backend: dial tcp 10.204.101.149:10250: i/o timeout

This happens most of the time, but not all of the time. Is there something I have to do before restarting port forwarding?

ERR_EMPTY_RESPONSE with hello Docker

Hi guys,
Looks like an amazing book !
I've been trying to launch the
docker run -d -p 8080:8080 --name hello cloudnatived/demo:hello command and then access http://localhost:8080 but I get the following error:

localhost didn’t send any data.
ERR_EMPTY_RESPONSE

Any idea ?
Thanks !

Helm 3

Could be useful to update the instruction on using helm to its latest stable version (ie. 3.0) that does not use tiller anymore?

Thanks so much for you pretty nice book.

Sops and helm usage proposal

Hi, thank you very much for writing a great book!

I'd like to make a proposal (or possibly question or discussion) on sops and helm usage for feeding an applications' secret file to k8s secret object.

Currently hello-sops example places staging-secrets.yaml and production-secrets.yaml inside of the demo chart.

k8s
 └── demo
     ├── <others are omitted for brevity>
     ├── Chart.yaml
     ├── production-secrets.yaml
     └── staging-secrets.yaml

However, the book suggests (in an applicable situation) making one single secret file (not k8s secrets object, but something like .env or production-secrets.yaml) as a strategy of avoiding secret duplication across applications.

Let's say I have multiple charts like demo1 and demo2. Also assume I have a secret file production-secrets.yaml, which has secrets for both demo1 and demo2. Unlike hello-sops, I want to keep it independent from each chart, so place it outside of the charts.

k8s
 ├── demo1
 │     ├── <omitted for brevity>
 │     └── Chart.yaml
 ├── demo2
 │     ├── <omitted for brevity>
 │     └── Chart.yaml
 └── production-secrets.yaml

However, .Files.Get cannot read an external file outside of the chart (There's a discussion though, helm#3276).

Thus, after decrypting production-secrets.yaml, I have to copy and paste the decrypted file under demo1 and demo2.

This is cumbersome.

Fortunately, using --set-file and .Values instead of .Files.Get can solve the problem.

For instance,

helm upgrade --install \
  --set-file mysecrets=./k8s/decrypted-production-secrets.yaml \
  production-demo ./k8s/demo 

# secrets.yaml in helm chart
data:
  mysecrets.yaml: {{ .Values.mysecrets | b64enc }}

helmfile supports --set-file as well. The below is same as the above.

releases:
- name: production-demo
  chart: k8s/demo 
  set: # --set-file mysecrets=./k8s/decrypted-production-secrets.yaml
  - name: mysecrets
    file: k8s/decrypted-production-secrets.yaml

I feel this way is more practical, as secrets are now "free" from charts.

I made a simple demonstration repo jjangga0214/k8s-sops-helm, so you might take a look.

How do you think of this?

Thanks.

apiVersion: extensions/v1beta1 errors

Hello there
I am writing to reference
demo/hello-k8s/k8s/deployment.yaml
The apiVersion is extensions/v1beta1 which when I try kubectl apply -f with this yaml file I get an error
error: unable to recognize "k8s/deployment.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"
I am using minikube on a MacMini

kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:47:22Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

I was able to resolve by changing the the apiVersion to
apiVersion: apps/v1
I can perform a pull request if you find that to be helpful, thank you I am enjoying the book :D

Sops example error

Hi,
I'm trying to run the sops example, but I get the following error:

Group 0: FAILED
  8252CFC3A36E12F7214687F44E66A6921AAADBBF: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Could not load secring:
      | open /Users/joachimhagege/.gnupg/secring.gpg: no such file
      | or directory; GPG binary error: exit status 2

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

Do I need to provide a specific key ?
Thanks.

Error: container has runAsNonRoot and image will run as root

Hi all,

deploying the hello-k8s simple demo app to our PKS kubernetes cluster shows this error in

C:\kubectl describe po demo-b579d8cb6-25lvv

Warning Failed 5s (x3 over 37s) kubelet, ab58d759-e5ad-4f5c-8a84-8039c0181301 Error: container has runAsNonRoot and image will run as root

What is wrong here in the image?

Regards,
Christian