noironetworks / openshift_vsphere_upi

Ansible playbooks for installing Openshift on vSphere UPI with ACI-CNI plugin

Openshift on Vsphere UPI, ACI-CNI plugin

Ansible playbooks for installing Openshift on Vmware user provisioned infrastructure with ACI-CNI plugin. This document is for Openshift version 4.6 and CoreOS 4.6. If you are installing Openshift version 4.5 please refer to ocp45 branch

Step 1 - acc-provision

• Provision ACI fabric using acc-provision utility.

  • Specify the flavor parameter value as 'openshift-4.6-esx'.
  • Specify an archive tar file for '-z' option, the archive file created will be required in the next steps

  Example acc-provision -a -c acc_provision_input.yaml -u admin -p ### -f openshift-4.6-esx -z manifests.tar.gz

  On successful execution, a portgroup with name <system_id>vlan<kubeapi_vlan> will be created under the distributed switch. This document will refer to this portgroup as api-vlan-portgroup.

Step 2 - VM Provisioning

• Download OCP46 OVA from Redhat site and import it. Specify api-vlan-portgroup as the port group for network interface.
• LoadBalancer: Provision a RHEL 8 VM with network interface connected to api-vlan-portgroup. This VM will be configured as loadbalancer for the openshift cluster.
• Orchestrator: Provision a RHEL 8 VM with network interface connected to api-vlan-portgroup.

Setp 3 - Configure the LoadBalancer

• Connect to the VM via console and configure basic network connectivity. Remember that the interface is a VLAN Interface for the <kubeapi_vlan>. Assign to the VM an IP in your Node Subnet
• Give the VM an hostname

Setp 4 - Configure the Orchestrator

• Register and subscribe: https://access.redhat.com/solutions/253273

• Enable ansible-2.9-for-rhel-8-x86_64-rpms repository: https://access.redhat.com/solutions/265523

• Update and install the required packages:

• Generate ssh keys and copy the ssh keys to loadbalancer

   ssh-keygen
   ssh-copy-id root@<LB_IP> 
   ssh-copy-id root@<Orchestrator_IP> YES to yourself :)
  

• Clone this repository and change directory to the git cloned directory.

• Install ansible module requirements. ansible-galaxy install -r requirements.yaml

• Edit group_vars/all.yml and hosts.ini file as per site requirements.

• perform basic validation of variable values using asserts.yml playbook ansible-playbook asserts.yml

• copy the archive file created by acc-provision to files directory with name as aci_manifests.tar.gz. Alternatively the file can be specified on in the default_aci_manifests_archive variable in the group_vars/all.yml file.

• Run setup playbook to configure this VM and the loadbalancer. ansible-playbook setup.yml

• Run oshift_prep playbook to generate openshift manifests and ignition files. ansible-playbook oshift_prep.yml

• Run create_nodes playbook to bring up the cluster. This playbook creates the bootstrap node, master and worker nodes. ansible-playbook create_nodes.yml

At this point, cluster creation has started, if auto_approve_csr option was not enabled, monitor the csr's pending and approve them for cluster creation to progress.

Delete

To delete the cluser, use delete_nodes playbook.

openshift_vsphere_upi's Issues

OpenShift Pull Secret better link

The best way to get the pull secret is to use this link:
https://cloud.redhat.com/openshift/install/pull-secret

The article provided in https://github.com/noironetworks/openshift_vsphere_upi/blob/master/group_vars/all.yml#L143 does not really works.

ACI CISCO + OKD 4.5 or 6

Staff,

We are trying to implement an openshift cluster OKD 4.5 or 6 + Fedora coreOS 33.20210217.3.0, using this repository as a base, we have direct access to the internet without a proxy.

We were able to initialize all the machines, bootstrap, masters and workers, but the deployment does not end, the masters are ready, but the only operators who went up were the Cloud-credential network, workers were not able to go into production, they are on the boot, trying to fetch your boot settings and we have no feedback on completing the bootstrap configuration (timeout)

have you already been successful in integrating ACI + OKD?

could you guide us about this implementation?

Thanks.

Compatibility with Ansible 2.13 and above

Due to a change in behaviour in how Templating works in ansible 2.13+ the generation of masters_api_ip_list and workers_api_ip_list will fail in the common role.

Our code can be re-written as:

- set_fact:
      masters_api_ip_list: "{{ masters_api_ip_list | default([]) + [ item.value.ip ] }}"
  with_dict: "{{ masters_vars.nodes }}"

- set_fact:
     workers_api_ip_list: "{{ workers_api_ip_list | default([]) + [ item.value.ip ] }}"
  with_dict: "{{ workers_vars.nodes }}"

DNS pods did not resolve

Dear all,

our setup:
ACI - 4.2(7t);
VMware - 7 u3;
in the vars file is set to not provision DNS on the LB machine;

We tried several times to install the OCP without success(ocp 4.13, ocp 4.12, ocp4.11). Almost all the Cluster Operators are OK, but some of them are receiving errors with the communication. Based on that we troubleshooted it and we figured out that they have a problem with the DNS pods. There is a problem with the DNS pods in openshift-dns namespace.

The DNS pods are in a RUNNING state.
When trying to execute on them "dig dig oauth-openshift.apps." it is timeout. But if we execute the same command from the nodes it is OK.

Are we missing something from your docs(https://www.cisco.com/c/en/us/td/docs/dcn/aci/containers/installation/openshift-on-vsphere/installing-openshift-4-13-on-vmware-vsphere.html)?

disk_size_MB is actually GB

In the group_vars/all.yml config file we set the VMs disk size with a variable called disk_size_MB this is then passed to the terraform vsphere_virtual_machine disk.size attribute.

The disk.size attribute is specified in GB, so the disk_size_MB variable should be renamed to disk_size_GB