fix(powerflex): upgrade powerflex to support k8s 1.30.x

Describe the bug
The current powerflex csi driver(v2.6.0) does not support kubernetes v1.30.x.

Upgrade it to the latest v2.11.0
(https://github.com/dell/csi-powerflex/releases/tag/v2.11.0)
to support kubernetes v1.30.3.

Versions (please complete the following information):

OS: Rocky Linux 8.10
Burrito: 1.3.0 to 2.1.0

kube-apiserver has a wrong --advertise-address

Describe the bug

The parameter --advertise-address in all kube-apiserver is set to
the first control node ip address.
It should be set to keepalived VIP address.

To Reproduce

Steps to reproduce the behavior:

Install kubernetes using ./run.sh k8s
See /etc/kubernetes/manifests/kube-apiserver.yaml.
It has --advertise-address=<the_first_control_node_ip_address>.

Expected behavior

The parameter --advertise-address should be set to keepalived VIP address.

Versions (please complete the following information):

OS: All
Burrito 2.1.0

local repo and registry should be on control nodes.

Is your feature request related to a problem? Please describe.
The local repository and registry pods are running on compute nodes when it is deployed.
They are the services for the platform and containers. So I think it belongs to control plane.

Describe the solution you'd like
Add nodeSelector in podSpec to run them on control nodes.

nodeSelector:
  node-role.kubernetes.io/control-plane: ""

kubernetes upgrade fails

Describe the bug

Upgrading from burrito 2.0.x to 2.1.0 is failed.

kubespray/roles/kubernetes/control-plane/tasks/kubeadm-upgrade.yml

The task 'Kubeadm | Upgrade first master' fails since other
parameters are not allowed when we use --config parameter.

We need to remove --config parameter.

roles/burrito.cert-manager/tasks/cert_manager.yml

The task 'Cert Manager | apply ca-issuer ClusterIssuer manifest' fails
since it tries to create a clusterissuer before cert-manager service is
upgraded and ready.

We need to try longer to create a clusterissuer.

To Reproduce

Steps to reproduce the behavior:

./run.sh k8s -e upgrade_cluster_setup=true

Expected behavior

Upgrading from 2.0.x should be successful.

Versions (please complete the following information):

OS: All
Burrito 2.1.0

Burrito images

Dear Sirs,

Are the Burrito images and/or the file: 'burrito_os_images.tar' publicly available?
Do you intent to make them publicly available?
Thank you in advance.

Best Regards,

Vasilios Pasias

ha.yml file typo problem

The hosts part of the ha.yml file needs to be modified

$ cat ha.yml

name: setup haproxy
hosts: kube-master
any_errors_fatal: true
roles:
- { role: burrito.haproxy, tags: ['haproxy', 'burrito']}
name: setup keepalived
hosts: kube-master
any_errors_fatal: true
roles:
- { role: burrito.keepalived, tags: ['keepalived', 'burrito']}
  ...

$ cat ha.yml

name: setup haproxy
hosts: kube_control_plane
any_errors_fatal: true
roles:
- { role: burrito.haproxy, tags: ['haproxy', 'burrito']}
name: setup keepalived
hosts: kube_control_plane
any_errors_fatal: true
roles:
- { role: burrito.keepalived, tags: ['keepalived', 'burrito']}

/etc/hosts issue when adding new nodes

When you add a new computer node, /etc/hosts does not generate an address for cinder.

new compute node

$ cat /etc/hosts
...
192.168.xx.xx control01.cluster.local control01

In this case, VM creation fails due to lack of connection refuse with cinder.

$ curl -v http://cinder.openstack.svc.cluster.local:8080
...

Failed to connect to cinder.openstack.svc.cluster.local port 8080: Connection refused
Closing connection 0
curl: (7) Failed to connect to cinder.openstack.svc.cluster.local port 8080: Connection refused

If you add the following information to /etc/hosts and restart the pod, it will be connected to the cinder and operate normally.

$ sudo vi /etc/hosts
...
192.168.20.11 control01.cluster.local control01 cinder.openstack.svc.cluster.local

$ curl -v http://cinder.openstack.svc.cluster.local:8080
...

Connection #0 to host cinder.openstack.svc.cluster.local left intact

Please check the above issue.

kube-apiservers use only the first etcd node cert and key

Describe the bug

The kube-apiserver on each control node uses the first etcd node certificate
and key files.
You can see this in /etc/kubernetes/manifests/kube-apiserver.yaml.

To Reproduce

Steps to reproduce the behavior:

Install kubernetes with ./run.sh k8s
See the /etc/kubernetes/manifests/kube-apiserver.yaml on each control node.

Expected behavior

Each kube-apiserver on the control plane uses each node's certificate and
key files.

For example, if there are 3 control nodes - control1, control2, and control3,
kube-apiserver on each node should use its own etcd node certificate and key.

control1's kube-apiserver manifest

$ sudo grep etcd-.*file /etc/kubernetes/manifests/kube-apiserver.yaml
        - --etcd-cafile=/etc/ssl/etcd/ssl/ca.pem
        - --etcd-certfile=/etc/ssl/etcd/ssl/node-control1.pem
        - --etcd-keyfile=/etc/ssl/etcd/ssl/node-control1-key.pem

control2's kube-apiserver manifest

$ sudo grep etcd-.*file /etc/kubernetes/manifests/kube-apiserver.yaml
        - --etcd-cafile=/etc/ssl/etcd/ssl/ca.pem
        - --etcd-certfile=/etc/ssl/etcd/ssl/node-control2.pem
        - --etcd-keyfile=/etc/ssl/etcd/ssl/node-control2-key.pem

control3's kube-apiserver manifest

$ sudo grep etcd-.*file /etc/kubernetes/manifests/kube-apiserver.yaml
        - --etcd-cafile=/etc/ssl/etcd/ssl/ca.pem
        - --etcd-certfile=/etc/ssl/etcd/ssl/node-control3.pem
        - --etcd-keyfile=/etc/ssl/etcd/ssl/node-control3-key.pem

Screenshots
If applicable, add screenshots to help explain your problem.

Versions (please complete the following information):

OS: All
Burrito 2.1.0 and earlier

Additional context
Add any other context about the problem here.

Unknown null interface creation issue.

Describe the bug
Unknown null interface creation issue.

To Reproduce
Steps to reproduce the behavior:

overlay_iface_name variable is null

Expected behavior
$ ip a s null
41: null: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether ee:d4:11:56:e0:7f brd ff:ff:ff:ff:ff:ff
inet6 fe80::ecd4:11ff:fe56:e07f/64 scope link
valid_lft forever preferred_lft forever

Screenshots

Versions (please complete the following information):

OS: Rocky Linux 8.7
Burrito 1.1.1

Additional context

Add netapp NFS managementLIF and dataLIF check in checklist playbook

Is your feature request related to a problem? Please describe.
NetApp nfs variables are defined in group_vars/all/netapp_vars.yml.
There are managementLIF and dataLIF ip addresses and it should be reachable from any burrito nodes.
There is no check for it .

Describe the solution you'd like

We can check if managementLIF and dataLIF is reachable from burrito nodes
when we run preflight playbook.

neutron-ovs-agent pod readiness probe failed

Installing and testing burrito testbed env.

When openstack's instance (VM) shutdown, the status of neutron-ovs-agent pod becomes 0/1(notready).

1.) shutdown instance

root@btx-0:/# openstack server list --all
+--------------------------------------+------+--------+---------------------------------------+--------+---------+
| ID                                   | Name | Status | Networks                              | Image  | Flavor  |
+--------------------------------------+------+--------+---------------------------------------+--------+---------+
| 585c39c9-c935-442e-880f-10f193fdcd56 | test | SHUTOFF | private-net=10.10.40.85, 172.30.1.140 | cirros | m1.tiny |
+--------------------------------------+------+--------+---------------------------------------+--------+---------+

2.) neutron ovs agent pod status

[clex@control01 ~]$ kubectl get po -n openstack -owide|grep ovs-agent
...
neutron-ovs-agent-default-hcwvm                0/1     Running            0                 26h     192.168.20.102   compute02   <none>           <none>
...
Events:
  Type     Reason     Age                   From     Message
  ----     ------     ----                  ----     -------
  Warning  Unhealthy  105s (x513 over 26h)  kubelet  Readiness probe failed:

It seems to be caused by neutron-ovs-agent not being able to remove the virtual tap interface when vm shutdown.

neutron@compute02:/$ cat /tmp/neutron-openvswitch-agent-readiness.sh
#!/bin/bash
...
[ -z "$(/usr/bin/ovs-vsctl show | grep error:)" ]

neutron@compute02:/$ /usr/bin/ovs-vsctl show | grep error:
                error: "could not open network device tap9b640a49-79 (No such device)"

Is it "openstack(yoga)'s problem?" or "kubernetes readiness.sh script's problem?"

ceph_purge fails to run when offline services are down

Describe the bug

I ran ./scripts/clean_openstack.sh to uninstall openstack and
./scripts/clean_k8s.sh to uninstall kubernetes.
Then when I run ./run.sh ceph_purge to uninstall ceph,
it stops and says "The offline flag is up but offline services are not up.".

The ceph_purge tasks do not need offline services.

To Reproduce
Steps to reproduce the behavior:

./scripts/clean_openstack.sh
./scripts/clean_k8s.sh
./run.sh ceph_purge

...
The offline flag is up but offline services are not up.
Run offline services - /home/clex/burrito/scripts/offline_services.sh -u
Local repo is NOT running.

Local registry is NOT running.

Expected behavior
The ceph_purge tasks should be run.

Versions (please complete the following information):

OS: All
Burrito: 2.1.0 and earlier

Add to BGP speaker dragent schedulring driver

When a neutron pod node dies, there is a phenomenon in which the BGP Spaker dragent disappears.
To resolve this issue, you must add bgp_drscheduler_driver to neutron.conf.
Please refer to the document below.
openstack docs : (https://docs.openstack.org/mitaka/config-reference/networking/networking_options_reference.html)

setting

node down

root@btx-0:/# openstack bgp speaker show dragents bgpspeaker
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.

<title>504 Gateway Time-out</title>

504 Gateway Time-out

nginx

node up

root@btx-0:/# openstack bgp speaker show dragents bgpspeaker
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
+--------------------------------------+-----------+-------+-------+

The previously scheduled driver was found to have disappeared when the node was up.
To solve this problem, i used the driver below.

$ vi kubespray/roles/burrito.openstack/templates/osh/neutron.yml.j2
...
neutron:
DEFAULT:
bind_host: 127.0.0.1
debug: True
router_distributed: True
core_plugin: ml2
global_physnet_mtu: {{ openstack_mtu }}
service_plugins: neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin,neutron.services.l3_router.l3_router_plugin.L3RouterPlugin
#service_plugins: router
l3_ha_network_type: vxlan
dhcp_agents_per_network: 2

 bgp_drscheduler_driver: neutron_dynamic_routing.services.bgp.scheduler.bgp_dragent_scheduler.StaticScheduler

I tried node down and up after applying the driver.

node down

node up

I think the above problem has been solved, so please check if improvement of burrito.

iorchard / burrito Goto Github PK

burrito's Introduction

Burrito

Introduction

License

burrito's People

Contributors

Stargazers

Watchers

Forkers

burrito's Issues

The hosts part of the ha.yml file needs to be modified

new compute node

Screenshots

Additional context

setting

node down

504 Gateway Time-out

node up

node down

node up

Recommend Projects

Recommend Topics

Recommend Org

Jobs