andremotz / nextcloud-kubernetes

nextcloud for kubernetes

Nextcloud on Kubernetes

nextcloud for kubernetes

For more background information check out my blog-post at https://www.andremotz.com/nextcloud-docker-on-kubernetes-cluster-ssl-certificates/

These YAMLs can be used on a Kubernetes-cluster to set-up a Nextcloud using MariaDB and Nginx as a SSL/TLS-Proxy. The YAMLs were tested on Ubuntu 18.04 but should be compatible with any Kubernetes-cluster.

Updates:

• 2019-12-02: Updated Deployments: added selector/matchLabels, changed apiVersion: apps/v1

Prerequisites:

• Installed Ubuntu 18.04
• Basic Docker & Kubernetes knowledge

Source: https://linuxconfig.org/how-to-install-kubernetes-on-ubuntu-18-04-bionic-beaver-linux

$ sudo apt update && sudo apt upgrade -y
$ sudo apt install docker.io
$ sudo systemctl enable docker
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
$ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
$ sudo apt install kubeadm
$ sudo swapoff -a
$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
$ kubectl taint nodes --all node-role.kubernetes.io/master-

At this place you should note down the shown kubeadm join-message in your console to be able to connect further Kubernetes-nodes in the future.

Extra-hint: Use the following in an extra-terminal to be able to see what the Kubernetes-cluster is doing $ watch -n 10 kubectl get deployment,svc,pods,pvc,pv,ing

Deployment + Service: MariaDB

As a user (not root) create a folder nc-deployment, download pre-defined MariaDB-descriptions, adjust it to your needs and deploy:

$ mkdir nc-deployment
$ cd nc-deployment
$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-deployment.yaml

$ nano db-deployment.yaml
--> change MYSQL_PASSWORD here
--> change MYSQL_ROOT_PASSWORD here
--> change db's HostPath here, which should be the absolute location of 'nc-deployment'/db-pv (eg /home/andremotz/nc-deployment/db-pv)

$ kubectl create -f db-deployment.yaml

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-svc.yaml
$ kubectl create -f db-svc.yaml

Deployment + Service: Nextcloud:

Next, download Nextcloud-descriptions, adjust them and deploy:

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-deployment.yaml

$ nano nc-deployment.yaml
--> change NEXTCLOUD_URL
--> change NEXTCLOUD_ADMIN_PASSWORD
--> change MYSQL_PASSWORD (the value you've entered before)
--> change html's hostPath (eg. to /home/andremotz/nc-deployment/nc-pv)

$ kubectl create -f nc-deployment.yaml

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-svc.yaml
$ kubectl create -f nc-svc.yaml

Create self-signed certificates

The OMGWTFSSL-Docker image offers easy-to-use certificate-creation. Here we are using only a Pod, not a Deployment. Once the certificates are created, the Pod will stop.

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/omgwtfssl-pod.yaml

$ nano omgwtfssl-pod.yaml
--> change SSL_SUBJECT to your server's name
--> change CA_SUBJECT to your mail-adress
--> change SSL_KEY to a proper filename
--> change SSL_CSR to a proper filename
--> change SSL_CERT to a proper filename
--> change cert's hostPath (eg. to /home/andremotz/nc-deployment/certs-pv)

$ kubectl create -f omgwtfssl-pod.yaml

Deployment + Service: Nginx reverse Proxy

One could already easily adjust the Nextcloud-service to publish HTTP-driven service. However we want to use a Nginx-instance in front of our Nextcloud to be able to use HTTPS-encryption. For the proxy we are not using a Deployment but a Pod, to be able to make use of standard HTTP/HTTPS-ports 80 & 443

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nginx.conf

$ nano nginx.conf
--> change server_name (two locations in the file!) to the server name you've provided before for SSL_SUBJECT
--> change ssl_certificate to the filename you've provide before for SSL_CERT
--> change ssl_certificate_key to the filename you've provide before for SSL_KEY

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/proxy-pod.yaml

$ nano proxy-pod.yaml
--> change cert's hostPath to the location you have provided before---> change nginx-config's hostpath to the location where you've stored nginx.conf before (eg. /home/andremotz/nc-deployment/nginx.conf)
--> change nginx-logs' hostpath to a proper location

$ kubectl create -f proxy-pod.yaml

Now you should be able to point your browser to https:// and see a new Nextcloud-instance, running on a super-hyper nextlevel-Kubernetes cluster, that you could use for further cool stuff 😉

Final hint

Disable swap permanently – add a comment to the /etc/fstab – swap entry like this:

#/swap.img none swap sw 0 0

That’s it!! 😉

nextcloud-kubernetes's Issues

mariadb failing to start due to write permission error

trusted_domains error

Hi,

I did everything like listed but if I wnat to access the nextcloud webpage ill get an error "access from an untrusted domain". ill tried to add my ip as an value in the nc-deployment.yaml but that doesnt resolve the problem

Issue Mounting "Nginx-Config": hostpath type check failed 'not a file'

Hi @andremotz ,

First of all, thanks for the great repo. I have followed your guide to deploye my next-cloud in kubernetes cluser. Everything went well; except I am unable to create pod for proxy.

pod/proxy is stuck at containercreating as you can see below:

pod/db-6f4cffddbd-tdhsf   1/1     Running             0          15m     10.44.104.2     node2   <none>           <none>
pod/nc-78c4bc454b-vkvwz   1/1     Running             0          13m     10.44.166.133   node1   <none>           <none>
pod/omgwtfssl             0/1     Completed           0          11m     10.44.166.134   node1   <none>           <none>
pod/proxy                 0/1     ContainerCreating   0          6m25s   <none>          node2   <none>           <none>

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE   SELECTOR
service/db           ClusterIP   10.108.104.243   <none>        3306/TCP   15m   app=db
service/kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP    4d    <none>
service/nc           ClusterIP   10.105.57.161    <none>        80/TCP     13m   app=nc

NAME                 READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES      SELECTOR
deployment.apps/db   1/1     1            1           15m   db           mariadb     app=db
deployment.apps/nc   1/1     1            1           13m   nc           nextcloud   app=nc

NAME                            DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES      SELECTOR
replicaset.apps/db-6f4cffddbd   1         1         1       15m   db           mariadb     app=db,pod-template-hash=6f4cffddbd
replicaset.apps/nc-78c4bc454b   1         1         1       13m   nc           nextcloud   app=nc,pod-template-hash=78c4bc454b

after further dignosing the pod; I have found this:

Namespace:    default
Priority:     0
Node:         node2/192.168.198.154
Start Time:   Sat, 15 Aug 2020 11:20:58 +0500
Labels:       app=proxy
Annotations:  <none>
Status:       Pending
IP:
IPs:          <none>
Containers:
  proxy:
    Container ID:
    Image:          nginx
    Image ID:
    Ports:          80/TCP, 443/TCP
    Host Ports:     80/TCP, 443/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:
      TZ:  Europe/Berlin
    Mounts:
      /etc/nginx/certs from certs (rw)
      /etc/nginx/nginx.conf from nginx-config (rw)
      /tmp/docker.sock from docker-socket (rw)
      /var/log/nginx from nginx-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-mbzcr (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  certs:
    Type:          HostPath (bare host directory volume)
    Path:          /home/nc-deployment/certs-pv
    HostPathType:
  docker-socket:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/docker.sock
    HostPathType:  Socket
  nginx-config:
    Type:          HostPath (bare host directory volume)
    Path:          /home/nc-deployment/nginx.conf
    HostPathType:  File
  nginx-logs:
    Type:          HostPath (bare host directory volume)
    Path:          /home/nc-deployment/nginx-logs
    HostPathType:
  default-token-mbzcr:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-mbzcr
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age                    From               Message
  ----     ------       ----                   ----               -------
  Normal   Scheduled    8m50s                  default-scheduler  Successfully assigned default/proxy to node2
  Warning  FailedMount  4m25s (x2 over 6m42s)  kubelet, node2     Unable to attach or mount volumes: unmounted volumes=[nginx-config], unattached volumes=[nginx-config nginx-logs default-token-mbzcr certs docker-socket]: timed out waiting for the condition
  Warning  FailedMount  2m7s                   kubelet, node2     Unable to attach or mount volumes: unmounted volumes=[nginx-config], unattached volumes=[certs docker-socket nginx-config nginx-logs default-token-mbzcr]: timed out waiting for the condition
  Warning  FailedMount  31s (x12 over 8m45s)   kubelet, node2     MountVolume.SetUp failed for volume "nginx-config" : hostPath type check failed: /home/nc-deployment/nginx.conf is not a file

And this is my proxy-pod.yaml file:

kind: Pod
metadata:
  labels:
    app: proxy
  name: proxy

spec:
  containers:
  - env:
    - name: TZ
      value: Europe/Berlin

    name: proxy
    image: nginx
    ports:
    - containerPort: 80
      protocol: TCP
      hostPort: 80
    - containerPort: 443
      protocol: TCP
      hostPort: 443

    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /etc/nginx/certs
      name: certs
    - mountPath: /tmp/docker.sock
      name: docker-socket
    - mountPath: /etc/nginx/nginx.conf
      name: nginx-config
    - mountPath: /var/log/nginx
      name: nginx-logs
  restartPolicy: Always
  volumes:
    - name: certs
      hostPath:
        path: "/home/nc-deployment/certs-pv"
    - name: docker-socket
      hostPath:
        path: "/var/run/docker.sock"
        type: Socket
    - name: nginx-config
      hostPath:
        path: "/home/nc-deployment/nginx.conf"
        type: File
    - name: nginx-logs
      hostPath:
        path: "/home/nc-deployment/nginx-logs"

I have tried to comment the hostpth type, changed it to the Directory, DirectoryOrCreate as well. but no success.

Can you please help me in this regard.