Scripts used to quickly build and teardown an OCP Lab on my Home Server (Whydah).
- 8 RHEL7.3 VMs running in KVM
- Registered and appropriate subscriptions attached
- required RPMs installed, including atomic-openshift-installer
- docker installed and storage configured
- ready to install the OpenShift cluster from the jump VM.
- Access to DNS server. I'm using a personal domain hosted on domains.google.com.
- Access to DHCP server. I'm using my home router and tie specific IP addresses to known mac addresses. ie: VMs always get the same IP address from DHCP.
- RHEL 7 KVM hypervisor
rhel-guest-image-7.3-35.x86_64.qcow2
(from https://access.redhat.com/downloads/)- 2 NICs on your hypervisor. I use an onboard NIC plus an inexpensive USB NIC.
Enable Ansible Repos
`sudo subscription-manager repos --enable rhel-7-server-ansible-2.5-rpms`
Install Ansible
`sudo yum -y install ansible`
Install libguestfs
`sudo yum -y install libguestfs-tools`
Edit env.sh
for your environment:
- VM_LIST - the names of the VMs to create
- DOMAIN - the domain name to use for the hosts (ie: batcluster)
- MACADDRESS - MAC addresses for your VMs (be unique)
- OCPDOMAIN - the domain name for the cluster (ie: ocp.btsaunde.com, *.apps.btsaunde.com)
- WORKSPACE, VMS - where VMs, etc are stored
- ISOS - where your ISOs can be found
- RHEL_IMAGE - your rhel-guest-image-7.3-35.x86_64.qcow2 is
- BRIDGE - which bridge to use. See Network Notes below
I am using nip.io for my DNS to point to the local addresses so it looks like this. eg:
$ nslookup jump.$IPADDRESS.$DOMAIN
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: jump.$IPADDRESS.$DOMAIN
Address: 192.168.1.107
You need to setup wildcard DNS entries for *.ocp.$OCPDOMAIN
and *.apps.$OCPDOMAIN
to point to lb.$IPADDRESS.$DOMAIN
in your DNS server. I am using Google Domains for this.
Tie these specific IP addresses defined in DNS to known mac addresses. ie: VMs always get the same IP address from DHCP.
Host | MAC Address | Assigned IP |
---|---|---|
jump.$DOMAIN | 52:54:00:42:B4:AD | 192.168.1.59 |
master0.$DOMAIN | 52:54:00:2C:C2:A0 | 192.168.1.107 |
master1.$DOMAIN | 52:54:00:AC:C6:E1 | 192.168.1.145 |
master2.$DOMAIN | 52:54:00:DE:6B:C4 | 192.168.1.16 |
lb.$DOMAIN | 52:54:00:96:FF:84 | 192.168.1.175 |
node1.$DOMAIN | 52:54:00:4A:22:9B | 192.168.1.245 |
node2.$DOMAIN | 52:54:00:4A:22:9C | 192.168.1.246 |
node3.$DOMAIN | 52:54:00:4A:22:9D | 192.168.1.248 |
I added a second NIC in the form of an inexpensive USB 3 NIC (enp35s0f3u2).
Here are the NetworkManager CLI commands to create another bridge (br0) and have enp35s0f3u2 bound (aka slaved) to it.
nmcli con add type bridge ifname br0
nmcli con show
# some will be yellow
nmcli -f bridge con show bridge-br0
# just take a look
nmcli con add type bridge-slave ifname enp35s0f3u2 master br0
ifup br0
nmcli con show
# all green
Then edit /etc/qemu-kvm/bridge.conf to add:
allow br0
Optional just finishing up libvirt config
virsh net-list --all
If you don't see a default
network entry from the previous
command, do this:
virsh net-define /usr/share/libvirt/networks/default.xml
virsh net-start default
virsh net-autostart default
You need to set the openshift_subscription_pool
for your own Red Hat account.
Use this command will find your pool id:
subscription-manager list --all --available --matches "*openshift*"
You also need to set your RHN Username and Password
Make variable.yml
look something like this:
openshift_subscription_pool: 8a85f98660c23a380160c2fa54a2300c
Create a vault to store your own Red Hat subscription username/password in variables. (ie: what you use on the Red Hat portal)
ansible-vault create vault.yml
Set your RHN credentials using the following format
vault_rhn_username: my-rhn-support-username
vault_rhn_password: secretpassword-for-rhn
Take a look at the resulting file and it should not have the variables in cleartext.
0-generate.sh
-- Create hosts and hosts.ocp based on your env.sh settings1-create.sh
-- Create qemu files for OS, container storage, OS config2-build.sh
-- Install VMs and attach disksstart-all.sh
-- boot them up3-keys.sh
-- push ssh keys around4-prep.sh
-- update the VMs with required packages, etc5-cluster.sh
-- copy files to jump VMs and remind the next steps
-
hypervisor# ssh [email protected] # password is redhat
-
jump# ssh-keygen
-
jump# bash ./3-keys.sh
-
jump# ansible-playbook -i hosts.ocp /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml
-
Once the cluster is created, ssh root@master0 and create a non-admin user:
# touch /etc/origin/master/htpasswd
# htpasswd /etc/origin/master/htpasswd someuser
The easiest way to get started is to point a browser to https://ocp.$OCPDOMAIN:8443/ (in my example, https://ocp.nozell.com:8443)
- Fix warning messages from ansible (replace sudo with become/become_user/become_method, service module, etc)
Based on the original work from Marc Nozell (irc: MarcNo)