Burrito is a security-hardened OpenStack on Kubernetes Platform.
Online documentation can be found at https://burrito.readthedocs.io/.
Burrito is licensed under Apache-2.0.
You can find the complete text in LICENSE.
Burrito - OpenStack on K8S platform
License: Apache License 2.0
Burrito is a security-hardened OpenStack on Kubernetes Platform.
Online documentation can be found at https://burrito.readthedocs.io/.
Burrito is licensed under Apache-2.0.
You can find the complete text in LICENSE.
Describe the bug
The current powerflex csi driver(v2.6.0) does not support kubernetes v1.30.x.
Upgrade it to the latest v2.11.0
(https://github.com/dell/csi-powerflex/releases/tag/v2.11.0)
to support kubernetes v1.30.3.
Versions (please complete the following information):
Describe the bug
The parameter --advertise-address in all kube-apiserver is set to
the first control node ip address.
It should be set to keepalived VIP address.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The parameter --advertise-address should be set to keepalived VIP address.
Versions (please complete the following information):
Is your feature request related to a problem? Please describe.
The local repository and registry pods are running on compute nodes when it is deployed.
They are the services for the platform and containers. So I think it belongs to control plane.
Describe the solution you'd like
Add nodeSelector in podSpec to run them on control nodes.
nodeSelector:
node-role.kubernetes.io/control-plane: ""
Describe the bug
Upgrading from burrito 2.0.x to 2.1.0 is failed.
The task 'Kubeadm | Upgrade first master' fails since other
parameters are not allowed when we use --config parameter.
We need to remove --config parameter.
The task 'Cert Manager | apply ca-issuer ClusterIssuer manifest' fails
since it tries to create a clusterissuer before cert-manager service is
upgraded and ready.
We need to try longer to create a clusterissuer.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Upgrading from 2.0.x should be successful.
Versions (please complete the following information):
Dear Sirs,
Are the Burrito images and/or the file: 'burrito_os_images.tar' publicly available?
Do you intent to make them publicly available?
Thank you in advance.
Best Regards,
Vasilios Pasias
$ cat ha.yml
name: setup haproxy
hosts: kube-master
any_errors_fatal: true
roles:
name: setup keepalived
hosts: kube-master
any_errors_fatal: true
roles:
$ cat ha.yml
name: setup haproxy
hosts: kube_control_plane
any_errors_fatal: true
roles:
name: setup keepalived
hosts: kube_control_plane
any_errors_fatal: true
roles:
When you add a new computer node, /etc/hosts does not generate an address for cinder.
$ cat /etc/hosts
...
192.168.xx.xx control01.cluster.local control01
In this case, VM creation fails due to lack of connection refuse with cinder.
$ curl -v http://cinder.openstack.svc.cluster.local:8080
...
If you add the following information to /etc/hosts and restart the pod, it will be connected to the cinder and operate normally.
$ sudo vi /etc/hosts
...
192.168.20.11 control01.cluster.local control01 cinder.openstack.svc.cluster.local
$ curl -v http://cinder.openstack.svc.cluster.local:8080
...
Please check the above issue.
Describe the bug
The kube-apiserver on each control node uses the first etcd node certificate
and key files.
You can see this in /etc/kubernetes/manifests/kube-apiserver.yaml.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Each kube-apiserver on the control plane uses each node's certificate and
key files.
For example, if there are 3 control nodes - control1, control2, and control3,
kube-apiserver on each node should use its own etcd node certificate and key.
control1's kube-apiserver manifest
$ sudo grep etcd-.*file /etc/kubernetes/manifests/kube-apiserver.yaml
- --etcd-cafile=/etc/ssl/etcd/ssl/ca.pem
- --etcd-certfile=/etc/ssl/etcd/ssl/node-control1.pem
- --etcd-keyfile=/etc/ssl/etcd/ssl/node-control1-key.pem
control2's kube-apiserver manifest
$ sudo grep etcd-.*file /etc/kubernetes/manifests/kube-apiserver.yaml
- --etcd-cafile=/etc/ssl/etcd/ssl/ca.pem
- --etcd-certfile=/etc/ssl/etcd/ssl/node-control2.pem
- --etcd-keyfile=/etc/ssl/etcd/ssl/node-control2-key.pem
control3's kube-apiserver manifest
$ sudo grep etcd-.*file /etc/kubernetes/manifests/kube-apiserver.yaml
- --etcd-cafile=/etc/ssl/etcd/ssl/ca.pem
- --etcd-certfile=/etc/ssl/etcd/ssl/node-control3.pem
- --etcd-keyfile=/etc/ssl/etcd/ssl/node-control3-key.pem
Screenshots
If applicable, add screenshots to help explain your problem.
Versions (please complete the following information):
Additional context
Add any other context about the problem here.
Describe the bug
Unknown null interface creation issue.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
$ ip a s null
41: null: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether ee:d4:11:56:e0:7f brd ff:ff:ff:ff:ff:ff
inet6 fe80::ecd4:11ff:fe56:e07f/64 scope link
valid_lft forever preferred_lft forever
Versions (please complete the following information):
Is your feature request related to a problem? Please describe.
NetApp nfs variables are defined in group_vars/all/netapp_vars.yml.
There are managementLIF and dataLIF ip addresses and it should be reachable from any burrito nodes.
There is no check for it .
Describe the solution you'd like
We can check if managementLIF and dataLIF is reachable from burrito nodes
when we run preflight playbook.
Installing and testing burrito testbed env.
When openstack's instance (VM) shutdown, the status of neutron-ovs-agent pod becomes 0/1(notready).
1.) shutdown instance
root@btx-0:/# openstack server list --all
+--------------------------------------+------+--------+---------------------------------------+--------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+---------------------------------------+--------+---------+
| 585c39c9-c935-442e-880f-10f193fdcd56 | test | SHUTOFF | private-net=10.10.40.85, 172.30.1.140 | cirros | m1.tiny |
+--------------------------------------+------+--------+---------------------------------------+--------+---------+
2.) neutron ovs agent pod status
[clex@control01 ~]$ kubectl get po -n openstack -owide|grep ovs-agent
...
neutron-ovs-agent-default-hcwvm 0/1 Running 0 26h 192.168.20.102 compute02 <none> <none>
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 105s (x513 over 26h) kubelet Readiness probe failed:
It seems to be caused by neutron-ovs-agent not being able to remove the virtual tap interface when vm shutdown.
neutron@compute02:/$ cat /tmp/neutron-openvswitch-agent-readiness.sh
#!/bin/bash
...
[ -z "$(/usr/bin/ovs-vsctl show | grep error:)" ]
neutron@compute02:/$ /usr/bin/ovs-vsctl show | grep error:
error: "could not open network device tap9b640a49-79 (No such device)"
Is it "openstack(yoga)'s problem?" or "kubernetes readiness.sh script's problem?"
Describe the bug
I ran ./scripts/clean_openstack.sh to uninstall openstack and
./scripts/clean_k8s.sh to uninstall kubernetes.
Then when I run ./run.sh ceph_purge to uninstall ceph,
it stops and says "The offline flag is up but offline services are not up.".
The ceph_purge tasks do not need offline services.
To Reproduce
Steps to reproduce the behavior:
...
The offline flag is up but offline services are not up.
Run offline services - /home/clex/burrito/scripts/offline_services.sh -u
Local repo is NOT running.
Local registry is NOT running.
Expected behavior
The ceph_purge tasks should be run.
Versions (please complete the following information):
When a neutron pod node dies, there is a phenomenon in which the BGP Spaker dragent disappears.
To resolve this issue, you must add bgp_drscheduler_driver to neutron.conf.
Please refer to the document below.
openstack docs : (https://docs.openstack.org/mitaka/config-reference/networking/networking_options_reference.html)
root@btx-0:/# openstack bgp speaker show dragents bgpspeaker
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | :-) |
+--------------------------------------+-----------+-------+-------+
root@btx-0:/# openstack bgp dragent list
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | :-) |
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
+--------------------------------------+-----------+-------+-------+
root@btx-0:/# openstack bgp speaker show dragents bgpspeaker
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.
root@btx-0:/# openstack bgp dragent list
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | XXX |
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
+--------------------------------------+-----------+-------+-------+
root@btx-0:/# openstack bgp speaker show dragents bgpspeaker
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
+--------------------------------------+-----------+-------+-------+
root@btx-0:/# openstack bgp dragent list
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | :-) |
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
+--------------------------------------+-----------+-------+-------+
The previously scheduled driver was found to have disappeared when the node was up.
To solve this problem, i used the driver below.
$ vi kubespray/roles/burrito.openstack/templates/osh/neutron.yml.j2
...
neutron:
DEFAULT:
bind_host: 127.0.0.1
debug: True
router_distributed: True
core_plugin: ml2
global_physnet_mtu: {{ openstack_mtu }}
service_plugins: neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin,neutron.services.l3_router.l3_router_plugin.L3RouterPlugin
#service_plugins: router
l3_ha_network_type: vxlan
dhcp_agents_per_network: 2
bgp_drscheduler_driver: neutron_dynamic_routing.services.bgp.scheduler.bgp_dragent_scheduler.StaticScheduler
I tried node down and up after applying the driver.
root@btx-0:/# openstack bgp speaker show dragents bgpspeaker
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | XXX |
+--------------------------------------+-----------+-------+-------+
root@btx-0:/# openstack bgp dragent list
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | XXX |
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
+--------------------------------------+-----------+-------+-------+
root@btx-0:/# openstack bgp speaker show dragents bgpspeaker
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | :-) |
+--------------------------------------+-----------+-------+-------+
root@btx-0:/# openstack bgp dragent list
+--------------------------------------+-----------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-----------+-------+-------+
| 66363166-b5f7-474d-ac88-940fe7c61704 | control01 | True | :-) |
| 6cb90f1a-2bd3-4b33-9572-41db7088f5bc | control02 | True | :-) |
| 3c9faf8c-c6a3-4f27-8be1-0118ab4b3050 | control03 | True | :-) |
+--------------------------------------+-----------+-------+-------+
I think the above problem has been solved, so please check if improvement of burrito.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.