nodeup will fail in nodes about kops

[zetaab]

one possible breaking #15829 which modifies the same part of the code

[zetaab]

@justinsb I can confirm that #15829 modified the behaviour in APIServerIPs and ConfigServer.servers. Do you have somekind of idea how we could get it working? The problem is that dns name for kops-controller does not exists in dns=none clusters at least in OpenStack.

[zetaab]

created new cluster to aws and openstack. AWS works, openstack not. The command is

./kops create cluster \
  --cloud openstack \
  --name jessetesti.k8s.local \
  --state ${KOPS_STATE_STORE} \
  --zones x,y,z \
  --network-cidr 10.2.0.0/16 \
  --image ubuntu-2004-081223-devops \
  --bastion \
  --dns=none \
  --control-plane-count=3 \
  --node-count=3 \
  --node-size m1.medium \
  --control-plane-size m1.medium \
  --etcd-storage-type solidfire \
  --topology private \
  --networking calico \
  --api-loadbalancer-type public \
  --os-octavia=true \
  --os-ext-net xx-nap \
  --os-ext-subnet ext-ha-v4 \
  --os-lb-floating-subnet ext-ha-v4 --kubernetes-version 1.29.1 --yes

and

./kops create cluster --name jesseaws2.k8s.local --dns=none --zones eu-north-1a,eu-north-1b,eu-north-1c --control-plane-count=3 --node-count=3 --node-size t3.small --kubernetes-version 1.29.1 --control-plane-size t3.small

cat /opt/kops/conf/kube_env.yaml 
APIServerIPs:
- 172.20.120.96
- 172.20.146.122
- 172.20.54.100
CloudProvider: aws
ClusterName: jesseaws2.k8s.local
ConfigServer:
  CACertificates: |
    -----BEGIN CERTIFICATE-----
    MIIC+DCCAeCgAwIBAgIMF6y145gxZShmQ1iPMA0GCSqGSIb3DQEBCwUAMBgxFjAU
    BgNVBAMTDWtxxxX4tTNotU=
    -----END CERTIFICATE-----
  servers:
  - https://172.20.120.96:3988/
  - https://172.20.146.122:3988/
  - https://172.20.54.100:3988/
InstanceGroupName: nodes-eu-north-1a
InstanceGroupRole: Node
NodeupConfigHash: cYaiGjBPTqbN1eCGvAJxjastnXGFNWRR2i2mUoTC3M0=

cat /opt/kops/conf/kube_env.yaml 
CloudProvider: openstack
ClusterName: jessetesti.k8s.local
ConfigServer:
  CACertificates: |
    -----BEGIN CERTIFICATE-----
    MIIC+DCCAeCgAwIBAgIMF6y2D8jb/2jJ9N3HMA0GCSqGSIb3DQEBCwUAMBgxFjAU
    BgNVBAMTDWt1YmVybmV0ZXMtY2EwHhcNMjQwMTIwMTU0ODU3WhcNMzQwMTE5MTU0
    ODU3Wxxxx+gL
    62ktwBmZ9w90b9Y1n+7tC5ujAyqcGIe7CEUOUavY4XBQQoXvAwpzJaY6FaIoZpJ+
    X+12JyIAXpHlmA4NV/7VjKkDyAWDncEOsk0ImWXDXB8L3xipCaUJrtI5LTU=
    -----END CERTIFICATE-----
  servers:
  - https://kops-controller.internal.jessetesti.k8s.local:3988/
InstanceGroupName: nodes-xxx
InstanceGroupRole: Node
NodeupConfigHash: /hwCnaFYXi1GUHTKbWsKqR/FyjIBBH73e0s4vhi1OrI=

so we can clearly see that this is the issue. I will next investigate why its not working in similar way in OpenStack

[zetaab]

earlier https://github.com/kubernetes/kops/blame/master/upup/pkg/fi/cloudup/apply_cluster.go#L1461 this function was containing apiserverAdditionalIPs but now I cannot see anything. I can actually see loadbalancer ip address, but in case of OpenStack we are interested of apiserver ips, which are not part of that array at all.

[zetaab]

it did not solve the whole issue

Jan 22 20:32:18 nodes-xx-p6tj9t nodeup[1175]: W0122 20:32:18.368505    1175 main.go:133] got error running nodeup (will retry in 30s): failed to get node config from server: Post "https://100.68.2.89:3988/bootstrap": tls: failed to verify certificate: x509: cannot validate certificate for 100.68.2.89 because it doesn't contain any IP SANs; Post "https://100.72.3.40:3988/bootstrap": tls: failed to verify certificate: x509: cannot validate certificate for 100.72.3.40 because it doesn't contain any IP SANs; Post "https://100.76.3.165:3988/bootstrap": tls: failed to verify certificate: x509: cannot validate certificate for 100.76.3.165 because it doesn't contain any IP SANs

[zetaab]

had old controlplanes that did not have correct certs in kops-controller