kelseyhightower / kubeadm-single-node-cluster

How to bootstrap a single-node Kubernetes cluster on Google Compute Engine using kubeadm.

kubeadm: Single Node Kubernetes Cluster

This tutorial will walk you through bootstrapping a single-node Kubernetes cluster on Google Compute Engine using kubeadm.

Tutorial

Create a single compute instance:

gcloud compute instances create kubeadm-single-node-cluster \
  --can-ip-forward \
  --image-family ubuntu-1704 \
  --image-project ubuntu-os-cloud \
  --machine-type n1-standard-4 \
  --metadata kubernetes-version=stable-1.8 \
  --metadata-from-file startup-script=startup.sh \
  --tags kubeadm-single-node-cluster \
  --scopes cloud-platform,logging-write

Enable secure remote access to the Kubernetes API server:

gcloud compute firewall-rules create default-allow-kubeadm-single-node-cluster \
  --allow tcp:6443 \
  --target-tags kubeadm-single-node-cluster \
  --source-ranges 0.0.0.0/0

Fetch the client kubernetes configuration file:

gcloud compute scp kubeadm-single-node-cluster:/etc/kubernetes/admin.conf \
  kubeadm-single-node-cluster.conf

It may take a few minutes for the cluster to finish bootstrapping and the client config to become readable.

Set the KUBECONFIG env var to point to the kubeadm-single-node-cluster.conf kubeconfig:

export KUBECONFIG=$(PWD)/kubeadm-single-node-cluster.conf

Set the kubeadm-single-node-cluster kubeconfig server address to the public IP address:

kubectl config set-cluster kubernetes \
  --kubeconfig kubeadm-single-node-cluster.conf \
  --server https://$(gcloud compute instances describe kubeadm-single-node-cluster \
     --format='value(networkInterfaces.accessConfigs[0].natIP)'):6443

Verification

List the Kubernetes nodes:

kubectl get nodes

NAME                          STATUS    ROLES     AGE       VERSION
kubeadm-single-node-cluster   Ready     master    35m       v1.8.0

The node version reflects the kubelet version, therefore it might be different than the kubernetes-version specified above.

Find out Kubernetes API server version:

kubectl version --short

Client Version: v1.8.0
Server Version: v1.8.0

Create a nginx deployment:

kubectl run nginx --image nginx:1.13 --port 80

Expose the nginx deployment:

kubectl expose deployment nginx --type LoadBalancer

Cleanup

gcloud compute instances delete kubeadm-single-node-cluster

gcloud compute firewall-rules delete default-allow-kubeadm-single-node-cluster

rm kubeadm-single-node-cluster.conf

kubeadm-single-node-cluster's Issues

Failed to create load balancer:

Error
User "system:kube-controller-manager" cannot create configmaps in the namespace "kube-system

Analysis:
This is due to an issue in kubeadm: kubernetes/kubeadm#425.
kubeadm version: 1.7.5

Next:
After apply the rbac fix from kubernetes/kubeadm#425. I got the "Cannot EnsureLoadBalancer() with no hosts" for kubectl describe svc/nginx.
possibly kubernetes/kubernetes#34866

calico-policy-controller is pending and get nodes shows no resources found

I am using your slightly adapted startup.sh script in a Ubuntu 17.04 instance running on AWS. I find that the calico-policy-controller pod shows pending, and get nodes shows that no resources are found. I'm not sure whether the few adjustments I've made for AWS are causing this to break, or what else I might be missing.

My hostname does not exist in DNS, but I added a fully qualified entry to /etc/hosts using the internal AWS IP of my instance.

I tried adding this to kubeadm.conf per the Calico docs about the pod subnet, with the same result:

networking:
  podSubnet: 192.168.0.0/16

I changed the cloudProvider in kubeadm.conf to aws and the Environment="KUBELET_EXTRA_ARGS=--cloud-provider=aws" in the systemd drop-in.

I am using my instance internal IP, but no external one, in place of the curl to Google Compute metadata.

Here is what I'm seeing:

# kubectl get pods --all-namespaces
NAMESPACE     NAME                                                 READY     STATUS    RESTARTS   AGE
kube-system   calico-policy-controller-1727037546-rbk5b            0/1       Pending   0          10m
kube-system   etcd-standalone.local                      1/1       Running   1          20s
kube-system   kube-apiserver-standalone.local            1/1       Running   1          20s
kube-system   kube-controller-manager-standalone.local   1/1       Running   1          20s
kube-system   kube-dns-2425271678-vx0bx                            0/3       Pending   0          10m
kube-system   kube-scheduler-standalone.local            1/1       Running   1          20s

# kubectl get nodes
No resources found.

Thank you for your awesome work on Kubernetes - presentations, examples and demos, Etc!

Cluster is not started

Hello, guys.
I am following your instructions, but the cluster is not started.
At syslog I found the following entries:

Jan 22 18:57:11 kubeadm-single-node-cluster kubelet[4401]: error: unable to load client CA file /etc/kubernetes/pki/ca.crt: open /etc/kubernetes/pki/ca.crt: no such file or directory

Any ideas what can it be?

UPD:

Manual startup.sh run at the node returned:

mexx@kubeadm-single-node-cluster:~$ sudo kubeadm init --config=kubeadm.conf
[init] Using Kubernetes version: v1.8.7
[init] Using Authorization modes: [Node RBAC]
[init] WARNING: For cloudprovider integrations to work --cloud-provider must be set for all kubelets in the cluster.
        (/etc/systemd/system/kubelet.service.d/10-kubeadm.conf should be edited for this purpose)
[preflight] Running pre-flight checks.
        [WARNING FileExisting-crictl]: crictl not found in system path
[preflight] Some fatal errors occurred:
        [ERROR KubeletVersion]: the kubelet version is higher than the control plane version. This is not a supported version skew and may lead to a malfunctional cluster. Kubelet version: "1.9.2" Control plane version: "1.8.7"
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

Hardcoded kubernetes-version?

Looks like

kubeadm-single-node-cluster/startup.sh

Line 42 in 21b0056

kubernetesVersion: stable-1.7

is hardcoding the value. It looks like the provided gcloud commands + startup.sh will bring the value to KUBERNETES_VERSION env var anyway:

kubeadm-single-node-cluster/startup.sh

Lines 47 to 50 in 21b0056

	KUBERNETES_VERSION=$(curl -s -H "Metadata-Flavor: Google" \
	http://metadata.google.internal/computeMetadata/v1/instance/attributes/kubernetes-version)
	sudo kubeadm init --config=kubeadm.conf

I'm not sure if KUBERNETES_VERSION env will override the hardcoded value in MasterConfiguration. The kubeadm docs say it doesn't:

Note: When providing configuration values using both a configuration file and flags, the file will take precedence. For example, if a file exists with:

In this case, could the hardcoded value actually be causing kubeadm to always provision stable-1.7 regardless of the version saved in gce metadata key?