OpenShift Origin Multi-Node Cluster

An OpenShift 3.6 cluster of 5 machine:

Machine	Hostname	IP	Description
Toolbox	toolbox.ocp.local.net	10.0.1.100	Ansible, Bind DNS, and NFS storage
Master	master.ocp.local.net	10.0.1.101	Cluster Master node
Infra	infra.ocp.local.net	10.0.1.110	Infra Node hosting Docker registry and HA Proxy router
Node1	node1.ocp.local.net	10.0.1.111	Worker node
Node2	node2.ocp.local.net	10.0.1.112	Worker node

Hardware Requirements

This demo environment requires a machine of:

4 cores
16 GB RAM
40 GB free disk space

Machine	CPU	Memory	Primary Disk	Secondary Disk
Toolbox	1	2048 MB	40 GB dynamic allocation	NA
Master	1	3072 MB	40 GB dynamic allocation	15 GB, dynamic, for Docker storage
Infra	1	3072 MB	40 GB dynamic allocation	20 GB, dynamic, for Docker storage
Node1	1	3072 MB	40 GB dynamic allocation	20 GB, dynamic, for Docker storage
Node2	1	3072 MB	40 GB dynamic allocation	20 GB, dynamic, for Docker storage

Note: If your machine has 8 GB RAM you can do the following:

1. Edit the machines-config.yml:

Remove node2 at the end of the file.

Adjust machines memory to be:

Machine Memory

Toolbox 512 MB

Master 2048 MB

Infra 2048 MB

Node1 2048 MB

2. Edit /ansible/ansible-hosts.yaml: remove node2 from [worker_nodes] group.

Machine	Memory
Toolbox	512 MB
Master	2048 MB
Infra	2048 MB
Node1	2048 MB

Prerequisites

The following steps are tested on a Windows host machine.

Install VirtualBox

Install VirtualBox 1.5 Check the comparability between Vagrant and VirtualBox first.
Download the Extension Pack and add it to VirtualBox: File --> Preferences --> Extensions --> Add

Install Vagrant

Install Vagrant. This environment has been tested with Vagrant 1.9.7
Install Vagrant VirtualBox Guest Plugin

vagrant plugin install vagrant-vbguest

Network

All the machines are configures with NAT and Host Only adapters.

Note: The network may not behave properly over VPN, please disable any VPN before running the cluster.

Cluster Provisioning

Clone or download this repository to your host machine.

Recommendation: It's recommended to change the SSK keys under keys directory.

Navigate to your local copy and run the following command

cluster init [parallel]

Note: The command will start the machines in the following order:

master, infra, node1, and node2

toolbox

Note: The second argument 'parallel' is optional. It directs vagrant to start machines in parallel when possible.

Note: As it's the first run of the machines, Vagrant will run provisioning scripts to install any required tools and configure the machines connectivity and networking.

Connecting to The Cluster

Test Network Connectivity

Firstly, try to ping the cluster machines from your host:

ping 10.0.1.100
ping 10.0.1.101
ping 10.0.1.110
ping 10.0.1.111
ping 10.0.1.112

Add Cluster DNS to Host Machine

Now, we need to cluster DNS to host in order to resolve machines hostnames:

Go to Network Connections
Select the VirtualBox Host-Only Network --> Properties --> Internet Protocol version 4 --> Properties
Make sure that the IP addressis 10.0.1.1, otherwise, try another adapter.
Add DNS server 10.0.1.100 and save.
Try to ping using machine hostname:

ping toolbox.ocp.local.net

Test the wildcard DNS:

ping XYZ.cloudapps.ocp.local.net

SSH Connection to Cluster Machines

Use your preferred SSH Client to connect to the machines. I personally recommend MobaXterm. Use the private key in the /keys directory to connect. Machines Users:

User	Password
root	vagrant
vagrant	vagrant

Installation and Configuration

Recommendation: I recommend that you take snapshots of the machines at this point, so you call roll back to them.

Connect to the Toolbox machine, and make sure that Ansible can reach to all the cluster machines:

ansible cluster -m ping

Docker

Install Docker and configure its storage:

ansible-playbook /vagrant/ansible/playbooks/setup-docker.yml

Pull Docker images prior to OpenShift installation:

ansible-playbook /vagrant/ansible/playbooks/populate-docker-registry.yml

Recommendation: Take snapshots of machines: master, infra, node1, and node1.

OpenShift

Install OpenShift Prerequisites:

ansible-playbook /vagrant/ansible/playbooks/openshift-pre-install.yml

Install OpenShift:

ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml

Configure OpenShift:

ansible-playbook /vagrant/ansible/playbooks/openshift-post-install.yml

Now, try to login to OpenShift Console from host machine https://master.ocp.local.net:8443 Accept the certificate and use admin/admin to login
Also try the Docker registry https://registry-console-default.cloudapps.ocp.local.net

Cluster Control

Start the Cluster:

cluster up [parallel]

Shutdown the Cluster:

cluster down

Next Enhancements

Enhance documentation.
Convert provisioning shell scripts to Ansible playbooks.
Use GlusterFs for storage.

samir-saad / openshift-origin-multi-node-cluster Goto Github PK